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Overview 



Over the past few decades, a postsecondary credential has become increasingly important in the labor 
market, and college attendance has become more common. Unfortunately, however, many students leave 
college before receiving a degree, particularly those who are academically underprepared for college- 
level work. Many postsecondary institutions operate learning communities to promote students’ involve- 
ment and persistence in college. Learning communities typically place groups of students in two or more 
linked courses with mutually reinforcing themes and assignments. They seek to build peer relationships, 
intensify connections to faculty, and deepen understanding of coursework. While learning communities 
are increasingly popular, little rigorous evidence on their effects exists. 

As part of MDRC’s multisite Opening Doors demonstration, Kingsborough Community College in 
Brooklyn, New York — a large, urban college with a diverse student population that includes many im- 
migrants — operated one such learning community program. The program placed freshmen in groups of 
up to 25 who took three classes together during their first semester: an English class, usually at the devel- 
opmental level; an academic course, such as health or psychology; and a one-credit orientation course. 
The program provided enhanced counseling and tutoring as well as a voucher for textbooks. 

Using a rigorous research design, MDRC assigned 1,534 freshmen, at random, either to a program group 
that was eligible for the learning community or to a control group that received the college’s standard 
courses and services. Analyses in this report show that: 

• The program improved students’ college experience. Students in the program group felt more 
integrated and more engaged than students in the control group. 

• The program also improved some educational outcomes while students were in the learning 
community program, but the effects diminished in subsequent semesters. Program group stu- 
dents, for example, attempted and passed more courses and earned more credits during their first se- 
mester. 

• The program moved students more quickly through developmental English requirements. Stu- 
dents in the program group were more likely to take and pass English skills assessment tests that are 
required for graduation or transfer. 

• The evidence is mixed about whether the program increased persistence. Initially the program did 
not change the rate at which students reenrolled. In the last semester of the report’s two-year - follow-up 
period, however, slightly more program group members than control group members attended college. 

These results are not the final word on Kingsborough’ s program: MDRC plans to continue tracking sam- 
ple members’ outcomes for at least another year. 
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Preface 



Community colleges are “the Ellis Island of American higher education,” according to 
the January 2008 report of the National Commission on Community Colleges. Serving nearly 
12 million students annually, they provide a pathway into the middle class for many low- 
income individuals, including people of color, immigrants, full- and part-time workers, and stu- 
dents who are the first in their families to attend college. However, the increased access to post- 
secondary education that community colleges offer has not always translated into individual 
success for students. As many as 60 percent of incoming students at community colleges re- 
quire at least one developmental (or remedial) course, and too many drop out before getting a 
credential — often because they never get beyond developmental classes. This report offers 
promising evidence that a one-semester “learning community” intervention can provide an early 
boost to freshmen, helping students move more quickly through developmental English re- 
quirements and earn more credits in their first semester. 

At Kingsborough Community College in Brooklyn, freshmen were placed in groups of 
up to 25 who took three classes together during their first semester: an English class, usually at 
the developmental level; an academic course, such as health or psychology; and a one-credit 
orientation course. The program also provided enhanced counseling and tutoring and a textbook 
voucher. Kingsborough’ s program is part of MDRC’s Opening Doors demonstration, which is 
testing interventions at six community colleges designed to help low-income students stay in 
school and succeed. Opening Doors is the first large-scale study of community college pro- 
grams to use a random assignment design, the “gold standard” of program evaluation. 

The report concludes that the jury is still out on whether learning communities improve 
students' persistence in school. Although educational outcomes improved for students while they 
were in the learning community program, these effects diminished in later semesters, as the con- 
trol group students caught up. However, at the end of the follow-up period — three semesters after 
the intervention — there was an indication that Opening Doors students were somewhat more 
likely to be enrolled in college. As a result of these promising findings, Kingsborough has ex- 
panded its learning communities program, with the goal of serving 80 percent of incoming fresh- 
men by 2010. Currently, the program reaches about 65 percent of incoming freshmen. 

MDRC plans to continue following the students at Kingsborough to find out whether 
the learning community intervention has lasting effects. In addition, with our partners at the Na- 
tional Center for Postsecondary Research, we have launched a multi-college demonstration of 
learning communities that will test variations of this promising model. 



Gordon Berlin 
President 
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Executive Summary 



Over the past few decades, a postsecondary credential has become increasingly impor- 
tant in the labor market, and college attendance has become more common. Unfortunately, 
however, many students leave college before receiving a degree, particularly those who are aca- 
demically underprepared for college -level work. Many postsecondary institutions operate learn- 
ing communities to promote students’ involvement and persistence in college. Learning com- 
munities typically place groups of students in two or more linked courses with mutually rein- 
forcing themes and assignments. They seek to build peer relationships, intensify connections to 
faculty, and deepen understanding of coursework. While learning communities are increas- 
ingly popular, little rigorous evidence on their effects exists. 

As part of the Opening Doors demonstration and evaluation project jointly under- 
taken by MDRC and the MacArthur Foundation-funded Network on Transitions to Adult- 
hood, six participating colleges operated innovative programs aimed at increasing students’ 
achievement and persistence and, eventually, their graduation rates and earnings. Kingsbor- 
ough Community College in Brooklyn, New York — a large, urban college in the City 
University of New York (CUNY) system — tested a program called Opening Doors Learn- 
ing Communities. The program placed freshmen, most of whom failed one or more of the 
skills assessment tests that all incoming students take, into groups of up to 25 who took 
three classes together during their first semester. It also provided enhanced counseling and 
tutoring as well as a voucher for textbooks. 

This report discusses the program’ s implementation and its effects on students up to 
two years after they entered the study. Using a rigorous research design, MDRC randomly 
assigned students either to a program group that was eligible for the learning community or 
to a control group that received standard college courses and services. Any subsequent sub- 
stantial differences in outcomes can be attributed to the Opening Doors program. 

In summary, the key findings from this report are: 

• The program improved students’ college experience. Students in the pro- 
gram group felt more integrated and more engaged than students in the con- 
trol group. 

• The program improved some educational outcomes while students were 
in the learning community, but the effects diminished later. Program 
group students passed more courses and earned more credits during their first 
semester. 
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• The program moved students more quickly through developmental 
(remedial) English requirements. Students in the program group were 
more likely to take and pass the college’ s English skills assessment tests that 
are required for graduation or transfer. 

• The evidence is mixed about whether the program increased persistence 
in college. Initially the program did not change the rate at which students re- 
enrolled. At the end of the report’s follow-up period, however, slightly more 
program group members than control group members attended college. 



How Was the Program Implemented? 

Opening Doors Learning Communities — operated between 2003 and 2005 — 
placed groups of freshmen into three linked classes: an English course, usually at the devel- 
opmental level; an academic course, such as health or psychology; and a one-credit orienta- 
tion course. The instructor of the orientation course provided enhanced counseling to stu- 
dents, and the program provided enhanced tutoring as well as a voucher for textbooks. 

The program was targeted to freshmen who planned to attend college full time dur- 
ing the day and who did not test into English as a Second Language. ESL students were ex- 
cluded because they participated in another learning community program. For the same rea- 
son, students in four “career majors” were excluded for the first year of the study. Over 
three-fourths of the students were under age 21 when they entered the study. Reflecting the 
diversity of the student body at Kingsborough, 38 percent identified themselves as black, 27 
percent as white, and 20 percent as Hispanic. Almost three-fourths of the students in the 
study reported that they or at least one of their parents were born outside the United States. 

The following key findings on the implementation of Kingsborough’ s learning 
communities program are based on interviews with and surveys of the college’s administra- 
tors, faculty, staff, and students. 

• The program’s key features were all in place when operations began, 
and they remained in place throughout the study. 

Despite a compressed planning period, Kingsborough’ s program was well imple- 
mented from the start. This achievement reflects the college administration’s strong com- 
mitment to the program and the study. 

• The learning communities varied in class size and the degree to which 
faculty integrated their courses. 
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Over four semesters, Kingsborough ran 40 learning communities for the study: 31 
with developmental English and 9 with college-level English. Owing to several challenges 
— including the difficulty of predicting how many students would test into each English 
level — class size varied from 6 to 25 students, with an average of 17. 

All instructors developed a new syllabus or revised their regular syllabus for the 
learning community, and all learning communities had some joint assignments across 
classes. The degree of integration across the courses varied, however, as did the frequency 
of joint assignments. The instructors in most learning communities met regularly to discuss 
student progress and coordinate assignments, but, in a minority of communities, the instruc- 
tors rarely met. Thus, the study provides a strong test of the structural features of the learn- 
ing community, but it may not fully detect the effects of tightly integrating course curricula. 



How Was the Impact of the Program Evaluated? 

As noted above, to determine the effect, or “impact,” of Kingsborough’ s program, 
MDRC assigned students, at random, to either a program group or a control group. Random 
assignment occurred just before students registered for classes. The study is tracking both 
groups over time to determine whether the learning community program results in better 
outcomes for students. Random assignment ensures that the motivation levels and personal 
characteristics of students in the two groups were similar when the study began; hence, any 
subsequent differences in outcomes can be attributed to the program. The study is estimat- 
ing the value added of Opening Doors, above and beyond what students normally receive. 
Kingsborough offers a rich array of academic programs and services, so the bar is set rela- 
tively high for the program to surpass. Also, the study examines whether the package of 
reforms and enhancements in Opening Doors at Kingsborough led to different outcomes, 
compared with standard classes and services. The study will not, however, disentangle the 
effects of each component. 



Did the Program Make a Difference? 

This report discusses the program’s impacts on a range of educational outcomes. 
The learning communities program directly affected students during their first semester at 
Kingsborough. Many higher education experts believe that students’ academic and social 
experiences during that first semester play a substantial role in their future success — that 
students who develop strong initial connections with the material they study, with other stu- 
dents, and with faculty are more likely to persist in college than students who do not. Also, 
at Kingsborough, students who make better progress in meeting their developmental Eng- 
lish requirements may be more motivated to stay in school. 
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This report presents impacts for the full research sample at Kingsborough (1,534 stu- 
dents) for up to two years after students entered the study. The key impact findings follow. 

• The program improved students’ experiences in college. 

When surveyed approximately a year after entering the study, students in the pro- 
gram group reported that they felt more integrated at school and were more engaged with 
their coursework, instructors, and fellow students and had a stronger sense of belonging 
than did control group students. They were more likely to say that their courses required 
critical thinking and that they had acquired valuable academic and work skills. Finally, they 
were more likely to rate their college experience as “good” or “excellent.” These findings 
strongly suggest that the learning community program provided a markedly different ex- 
perience for students. These results are similar to findings from some past studies of learn- 
ing communities. 

• The program improved several educational outcomes for students dur- 
ing the semester that they participated in the learning community. 

MDRC examined students’ academic performance by using transcript data from 
Kingsborough. Newly enrolling students were randomly assigned for the study just before 
the start of their first semester in college (enrollment occurred in four different but contigu- 
ous semesters: fall 2003, spring 2004, fall 2004, and spring 2005). The first semester that 
each student was in the study is called the “program semester.” 

Figure ES . 1 illustrates some key outcomes during the program semester. The solid 
bars show the average outcomes for program group members, and the white bars show the 
averages for control group members. The difference between each pair of bars represents 
the program impact, if any, and asterisks indicate whether an impact is “statistically signifi- 
cant,” meaning that it is unlikely to be due to chance. (All impacts discussed in the Execu- 
tive Summary are statistically significant.) As the two sets of bars at the left of the figure 
show, students in the program group attempted and passed about half a course more at 
Kingsborough during their first semester in the study than control group students did. They 
also earned almost one more “developmental credit” (called an “equated credit” at Kings- 
borough). Developmental courses do not earn college credit, but they do count in determin- 
ing whether a student is attending school full time. Program group members were also more 
likely to pass all their courses during the first semester (not shown). 

• The positive effects on educational outcomes diminished in later semes- 
ters of the two-year follow-up period. 

The program generated a small increase in the number of credits attempted and 
earned in the first postprogram semester, but the effects dissipated later. By the end of the 
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The Opening Doors Demonstration 
Figure ES.l 

Key Impacts During the Program Semester 
Kingsborough Community College Report 



9 1 
8 - 




Courses attempted Courses passed 




Regular credits earned Developmental credits 

earned 



Impact 

SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



two-year follow-up period, program group members earned an average of two and a half credits 
more at Kingsborough than control group members. (This gain is primarily due to impacts dur- 
ing the program semester and, to some extent, during the first postprogram semester.) 

• Opening Doors Learning Communities helped students move more 
quickly through the college’s developmental English requirements. 

A goal of Kingsborough’ s program was to help students more quickly complete de- 
velopmental requirements and progress to college -level English. To enroll in the college- 
level course at Kingsborough, students must first pass the CUNY reading and writing skills 
assessment tests. (Students must pass the reading, writing, and math skills assessment tests 
in order to transfer to a four-year CUNY institution.) Students take the skills 
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The Opening Doors Demonstration 
Figure ES.2 

Impacts on English Skills Assessment Tests 
Kingsborough Community College Report 




Attempted Passed reading Attempted Passed writing Attempted Passed both 
reading test test writing test test either English English skills 

skills assessment 

assessment test tests 



Impact 



SOURCE: MDRC calculations from City University of New York skills assessment test data. 

NOTES: Outcomes include data from the program semester through the second postprogram semester. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



tests prior to enrolling; if they do not pass the reading and writing tests, they typically can 
retake them only after they pass specific developmental English courses. 

Figure ES.2 shows the proportion of the two research groups who took the tests dur- 
ing their first three semesters in the study and passed the tests by the end of that period (in- 
cluding students who passed the tests before starting their freshman year). The program in- 
creased the proportion of students who attempted and passed the tests. Although not illus- 
trated in the figure, most of these impacts are driven by effects in the first (program) semes- 
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ter. It is notable, however, that the control group members had not “caught up” in terms of 
test taking and passing by the end of the follow-up period. 

MDRC also examined progression through English courses for different subgroups 
of the research sample. Among the subset of the sample who failed both English skills as- 
sessment tests before starting their freshman year, program group members were more like- 
ly than their control group counterparts to enroll in developmental English during their first 
two semesters. Program group members who failed one of the tests before entering college 
were also more likely to enroll in developmental English during their first semester and 
were more likely to enroll in and pass college-level English during their first two semesters. 
The program did not affect progression through English courses among students who had 
passed both English assessment tests before starting their freshman year. 

• So far, the evidence is mixed about whether Kingsborough’s program 
increases student persistence in college. 

A central goal of all the Opening Doors programs is to increase persistence in col- 
lege. Initially, Kingsborough’s program did not change the rate at which students re- 
enrolled. In the last semester of the follow-up period, however, a difference emerged: 53 
percent of the program group registered for at least one course that semester at Kingsbor- 
ough, compared with 48 percent of the control group. Data from the National Student Clear- 
inghouse, which provides enrollment information at most colleges in the nation, shows a 
similar effect on persistence emerging that semester. MDRC plans to continue tracking out- 
comes to see if this effect remains. 



What Are the Implications of the Results? 

Opening Doors Learning Communities at Kingsborough substantially improved 
students’ experiences in college and some key educational outcomes while they were in the 
program, but, for the most part, the effects did not persist. MDRC plans to track sample 
members’ outcomes for at least three years after their random assignment to the study to 
determine the longer-term effects on their academic performance, persistence, and gradua- 
tion as well as on their later employment rates and earnings. Thus, the results in this report 
are not the last word on Kingsborough’s program. That said, the findings point to the fol- 
lowing conclusions. 

• Kingsborough’s learning community model shows promise as a strategy 
to help students move through developmental education. 

Many students begin at community colleges academically underprepared for col- 
lege-level work. Research shows that approximately 60 percent of freshmen beginning at 
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community college need at least one developmental-level course. 1 Students with very low 
skill levels can spend a year or more in developmental courses, and many leave school be- 
fore completing developmental requirements. A key challenge is how to help students meet 
these requirements so they can eventually complete college. 

This report's findings suggest that Kingsborough' s Opening Doors Learning Com- 
munities model is one strategy that college administrators could consider. Students in the 
program group were more likely to pass the English skills assessment tests — the gateway 
to college-level English. Also, program group students who failed one of the tests before 
entering school were more likely to take and pass college-level English during the follow- 
up period. It is important to highlight that Kingsborough' s program included English in the 
learning community. As a result, students in the program group were required to take Eng- 
lish, and, as discussed above, the program substantially increased the proportion of students 
who took developmental English. Because students must pass developmental classes before 
retaking the assessment tests, this program feature is central to the impact on test taking and 
passing. 



• The results from Kingsborough suggest that the jury is still out on 
whether learning communities improve students’ persistence. 

As noted, Kingsborough' s program did not increase students’ retention in college 
until the third postprogram semester, and MDRC will collect more follow-up data to deter- 
mine whether the effect continues. At this juncture, however, it is worthwhile to consider 
the absence, so far, of a strong program effect on retention. 

Kingsborough’ s program was based on the hypothesis that a more engaging and 
successful first semester would lead to more successful future semesters and higher rates of 
retention. One may wonder, however, how much change in college attendance is reasonable 
to expect from a one-semester program. Individuals make life choices, including whether or 
not to remain in college, based on myriad factors — many outside the college environment. 

Even if Kingsborough’ s program does not lead to substantial retention effects, it 
could still generate increases in degree receipt, transfer, and other longer-term outcomes, 
since the program group students who were still enrolled at the end of the report's follow- 
up period are somewhat further along in school than the control group members. 

• Enhanced services that last longer than one semester may have a more 
substantial effect on students. 



'Clifford Adelman, Principal Indicators of Student Academic Histories in Postsecondary Education, 
1972-2000 (Washington, DC: U.S. Department of Education, 2004). 
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Kingsborough’s program lasted one semester. The college’s administrators decided 
that there was no practical way to maintain the linked-course structure after the first semes- 
ter, since students needed and wanted to take a variety of different courses in subsequent 
semesters. Also, the program was designed on the assumption that students’ early experi- 
ences at college influence their later success, and administrators believed that students 
should transition into the regular college community as quickly as possible. 

The question of how long a learning community program should continue is compli- 
cated. That said, the results from the Kingsborough study suggest that participating in a learning 
community program for more than one semester may yield more substantial effects — the posi- 
tive effects on academic outcomes were by far the largest during the first semester. If the op- 
tions of a multiple-semester learning community or participating in a different learning commu- 
nity after the first semester are not possible, colleges could offer other kinds of enhanced ser- 
vices in later semesters, such as intensive counseling or more financial support. 

It is worth noting that, in some of the other sites in the Opening Doors demonstra- 
tion, the early results follow a similar pattern: Effects are largest when students receive en- 
hanced services, and they diminish or even disappear after the services end. 

* * * 

The study at Kingsborough is of a specific program model, targeted to a certain 
group of students, in a particular setting. Other learning community models, target groups, 
and institutional settings may well lead to different results. Another rigorous study, the 
Learning Communities demonstration, was launched in 2006 and is using random assign- 
ment to test the effects of learning communities in up to six colleges or universities. The 
demonstration is part of the National Center for Postsecondary Research, funded by the 
U.S. Department of Education. 
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Chapter 1 

Introduction 



This report presents results from a rigorous study of a learning communities pro- 
gram for incoming freshmen at Kingsborough Community College in Brooklyn, New York. 
Operated from 2003 to 2005 as part of the multisite Opening Doors demonstration, Kings- 
borough’ s learning communities program placed groups of up to 25 students each in three 
linked classes. The program also provided enhanced counseling and tutoring and a voucher 
to pay for textbooks. This report presents information on how the Opening Doors program 
was implemented at Kingsborough and what its effect are, up to two years after students 
entered the study. This is the first in a series of reports that will present detailed findings 
from all the Opening Doors demonstration sites. 

This report updates and supplements the early results from the Kingsborough study 
that were presented in a report in 2005 . 1 The 2005 report, which examined the first quarter 
of the sample members to enter the study, showed that the Opening Doors program led to 
better academic outcomes during students’ first semester at Kingsborough, including pass- 
ing more courses and increased completion of developmental English requirements a year 
after students entered the study. 

This chapter begins with an overview of the Opening Doors demonstration. It then 
focuses specifically on the study at Kingsborough, describing the college and its Opening 
Doors program model. The chapter concludes with a brief description of the contents of the 
rest of this report. 



Overview of the Opening Doors Demonstration and Evaluation 

This section discusses the importance of community colleges as a focus of study 
and provides some information about the development of the Opening Doors demonstra- 
tion. It also describes the study’s research design, guiding conceptual model, and key re- 
search questions. 

Why Focus on Community Colleges? 

Community colleges make higher education affordable and accessible to virtually 
anyone seeking the opportunity. Today, about 1,200 community colleges serve nearly 12 



'Bloom and Sommo (2005). 
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million students. Almost half of all college students nationwide attend a community col- 
lege. 2 Compared with four-year institutions, community colleges enroll more students of 
color and more low-income students. They are also more likely to enroll working adults and 
parents. 3 

Community colleges prepare students for transfer to four-year colleges and univer- 
sities, and they provide training in a wide variety of occupations. As shown in Figure 1.1, 
recent data from the U.S. Bureau of the Census indicate that, in 2005, an adult with an asso- 
ciate’s degree earned an average annual income of about $38,000, compared with only 
about $29,000 for an adult with a high school diploma. 4 Given the widening earnings gap 
between individuals with a postsecondary credential and individuals with a high school di- 
ploma, community colleges represent a potential pathway out of poverty and into the mid- 
dle class. 

Unfortunately, although many people attend community colleges, only a minority of 
students end up receiving a degree. The U.S. Department of Education reported that only 
about one-third of students who enter community college with the intention of earning a 
degree accomplish this goal at any institution of higher education within a six-year period. 5 
Completion rates are particularly low for students who begin college academically under- 
prepared and must take developmental-level courses. Recognizing the importance of higher 
education today, college administrators, policymakers, and researchers are searching for 
effective strategies to help community college students stay in school and succeed. 

The Opening Doors Programs 

Building on previous efforts to learn about factors that affect community college stu- 
dents’ enrollment and completion, MDRC, with support from a consortium of funders, launched 
the Opening Doors demonstration in 2003. A review of prior research and focus groups with 
past, current, and potential community college students 6 revealed some key factors that hinder 
students’ progress: underpreparation for college -level work; the challenges of juggling school, 
work, and family; and institutional barriers, such as inadequate support services and insufficient 
financial aid. Opening Doors is testing three promising strategies that colleges could adopt to 
address these factors and increase student achievement and persistence: 



2 American Association of Community Colleges Web site. 

3 Horn and Nevill (2006). 

4 The data shown in the figure are correlational; they do not indicate that a certain credential causes a 
certain level of earnings. 

5 U.S. Department of Education (2002). 

6 See Matus-Grossman and Gooden (2002) for a discussion of the focus group results. 
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The Opening Doors Demonstration 
Figure 1.1 

Average Annual Earnings, by Educational Attainment: Adults, Nationwide, 2005 
Kingsborough Community College Report 



Bachelor's degree 



Associate's degree 



High school diploma 



Without a high school 
diploma 




SOURCE: U.S. Bureau of the Census (2007). 



Annual Earnings 



• Curricular and instructional innovations, including learning communities 
in which students take blocks of classes with the same group of peers, cus- 
tomized instructional support, academic instruction for students on academic 
probation, and enhanced orientation courses to help students navigate the col- 
lege experience 

• Enhanced student services, including stronger, more personalized academic 
advisement, career counseling, and tutoring 

• Supplementary financial aid, such as special scholarships or money di- 
rected to specific education-related costs, like vouchers for textbooks 
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After a nationwide search, MDRC selected six community colleges in four states to 
participate in the demonstration and evaluation. MDRC worked with each college to de- 
velop an Opening Doors program that combined two or three of the strategies listed above. 
Table 1.1 lists the participating colleges and provides a brief summary of their Opening 
Doors programs and the groups of students who were targeted and served. Findings from 
the evaluation yield information about how to improve the rate of student success and will 
speak to the focus of the Spellings Commission — the U.S. Secretary of Education's Com- 
mission on the Future of Higher Education — on the need for investments that lead to better 
student outcomes. 

As noted above, this report focuses on Kingsborough Community College, which, 
in the fall of 2003, was the first demonstration site to begin operating its Opening Doors 
program. A later section of this chapter provides some information on Kingsborough and its 
Opening Doors program. 

The Research Design and Conceptual Model 

The evaluation of the Opening Doors programs is being conducted by MDRC, a 
group of scholars who are part of the MacArthur Foundation-funded Research Network on 
Transitions to Adulthood, and an expert on the relationship between education and health at 
Princeton University. 7 To measure the effects of Kingsborough’ s program, as well as the 
effects of the other colleges’ programs, the evaluation is using a random assignment re- 
search design, a first in large-scale community college research. At each college, students 
who met the site’s eligibility criteria and agreed to take part in the study were assigned, at 
random, either to a program group that received enhanced Opening Doors services or to a 
control group that received the college’s standard services. The study is tracking both 
groups over time to find out whether the enhanced services result in better outcomes for 
students. Random assignment ensures that the motivation levels and personal characteristics 
of students in the program group and control group were similar at the beginning of the 
study; hence, any subsequent substantial differences in educational or other outcomes can 
be attributed with a high level of confidence to Opening Doors. The study is estimating the 



7 Members of the Research Network on Transitions to Adulthood are Gordon L. Berlin (MDRC), 
Mark Courtney (University of Washington), Sheldon Danziger (University of Michigan), Connie A. 
Flanagan (Pennsylvania State University), Frank F. Furstenberg (University of Pennsylvania), Vonnie C. 
McLoyd (University of North Carolina, Chapel Hill ), Wayne Osgood (Pennsylvania State University), 
Jean E. Rhodes (University of Massachusetts, Boston), Cecilia E. Rouse (Princeton University), Ruben G. 
Rumbaut (University of California, Irvine), Richard Settersten (Oregon State University), and Mary C. 
Waters (Harvard University). Christina Paxson of Princeton University is leading the evaluation compo- 
nent focused on health outcomes. 
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The Opening Doors Demonstration 



Table 1.1 



Opening Doors Programs and Target Groups 



Kingsborough Community College Report 



Site Brief Program Description 

Chaffey College College Survival Skills and Enhanced 

Rancho Cucamonga, California Student Services: Students took a one- 

semester guidance course that provided 
instructional support as well as advising; 
students were required to visit the col- 
lege’s Success Centers, which provided 
extra academic support. 



Target Group 

Students ages 18-34 on aca- 
demic probation who earned 
fewer than 35 credits and who 
either had a cumulative grade 
point average below 2.0 (C) or 
did not complete at least half the 
courses in which they enrolled. 



Delgado Community College 
and Louisiana Technical Col- 
lege-West Jefferson 

New Orleans area, Louisiana 



Kingsborough Community 
College 

Brooklyn, New York 



Lorain County Community 
College 

Elyria, Ohio 



Owens Community College 

Toledo, Ohio 



A Scholarship Predicated on Academic 
Performance: Students were eligible for 
$1,000 scholarship for each of two semes- 
ters; scholarship was tied to maintaining 
at least half-time enrollment and a grade 
point average of 2.0 (C). 

Learning Communities and a Book 
Voucher: Groups of students took three 
linked courses together; students received 
enhanced advising and tutoring and 
vouchers to pay for textbooks. 

Enhanced Student Services and a Mod- 
est Scholarship: Students were assigned 
to an Opening Doors adviser with a small 
caseload with whom they were expected 
to meet frequently; students had access to 
designated contact in financial aid office; 
students were eligible for $150 scholar- 
ship for each of two semesters, paid after 
mandatory meetings with adviser. 

Enhanced Student Services and a Mod- 
est Scholarship: Students were assigned 
to an Opening Doors adviser with a small 
caseload with whom they were expected 
to meet frequently; students had access to 
designated contact in financial aid office; 
students were eligible for free one-on-one 
tutoring and for $150 scholarship for each 
of two semesters, paid after mandatory 
meetings with adviser. 



Parents ages 18-34 whose family 
income was below 200 percent 
of the federal poverty level. 



Incoming freshmen ages 17-34 
who planned to attend college 
full time. 



Students ages 18-34 whose fam- 
ily income was below 250 per- 
cent of the federal poverty level 
and who either were incoming 
freshmen or had completed 
fewer than 13 credits and had a 
history of academic difficulties. 



Students ages 18-34 whose fam- 
ily income was below 250 per- 
cent of the federal poverty level 
and who either were incoming 
freshmen or had completed 
fewer than 13 credits and had a 
history of academic difficulties. 



SOURCE: MDRC field research data. 
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value added of Opening Doors, above and beyond what students at the colleges normally 
receive. 



In designing the evaluation and developing the key research questions, MDRC and 
its research partners considered the basic conceptual model illustrated in Figure 1.2. The 
model provides a framework for linking the Opening Doors reforms to various outcomes 
that are important for a successful educational experience and transition to a better life. As 
illustrated in the figure, the Opening Doors reforms in curriculum and instruction, student 
services, and financial aid theoretically affect some key “early” educational or academic 
outcomes, while the program is operating (such as the number of credits completed and 
academic performance), as well as some “later” educational outcomes, including semester- 
to-semester persistence, graduation, and transfer to another postsecondary institution. 

As Figure 1.2 illustrates, the effects on the educational outcomes, if they are posi- 
tive and strong, would in turn lead to improved labor market outcomes in the longer term, 
including better jobs and higher earnings and enhanced well-being. Some of the social, per- 
sonal, and health outcomes (such as social networks or sense of self) might be affected in 
the short term, while others (such as physical and mental health) are more likely to be af- 
fected only in the longer term. 

The model illustrated in Figure 1.2 shows some basic relationships and, as noted 
above, helped guide the design of the study. As Opening Doors evolved, it became clear 
that some of the relationships observed in the study may be a bit more complicated. For ex- 
ample, in a learning communities program, such as the one tested at Kingsborough, groups 
of students take classes together as a cohort. This, in itself, could lead to changes in social 
networks, connections with peers, and similar outcomes. In fact, as discussed below, social 
changes such as these are central to the theory behind why learning communities might help 
students succeed. The relationships, in other words, are not all linear and one-directional, as 
illustrated in the simplified figure. This report examines a range of educational and well- 
being outcomes. Chapters 4 and 5 provide more information on the hypotheses about the 
likely effects of Kingsborough’ s program. 

Key Research Questions 

The Opening Doors evaluation includes three main components: an implementation 
analysis, an impact analysis, and a cost analysis. The key research questions in the imple- 
mentation analysis follow: 

• Were the enhanced services sufficiently distinct from the services available 
to the study’s control group to constitute a “fair test” of the intervention? 
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The Opening Doors Demonstration 
Figure 1.2 

Basic Conceptual Model for Evaluating the Effects of Opening Doors Programs 
Kingsborough Community College Report 
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• What services were provided as part of the Opening Doors programs? What 
was their quality and intensity, and how might the programs help students? 

• How were the programs managed and operated, and how were the services 
delivered? 

Chapter 3 of this report addresses these implementation questions for Kingsborough’s 
Opening Doors program. 

As discussed above, the random assignment design provides a rigorous way to esti- 
mate the effects of the enhanced programs on various outcomes of interest. Reflecting the 
outcomes shown in Figure 1.2, the key research questions in the analysis of the programs’ 
effects, or impacts, follow: 

1 . Do the Opening Doors enhancements in curriculum, student services, and fi- 
nancial aid in community colleges lead to more positive early educational 
outcomes — including completing more credits and earning better grades — 
compared with standard college courses and services? (The study is measur- 
ing the early educational outcomes only at the Opening Doors colleges, not 
other postsecondary institutions. See Chapter 2 for more detail on the data 
sources used in this report and what they measure.) 

2. Do the enhancements lead to more positive later educational outcomes, in- 
cluding higher rates of persistence in school, of degree attainment, and of 
transfer to four-year institutions? 

3. Do the enhancements or the resulting positive educational effects have a 
positive impact on students’ personal development, social networks, civic 
participation, and health behaviors? 

4. Do the enhancements or the resulting positive educational effects impact stu- 
dents’ success in the labor market? 

Chapter 4 provides some answers from the study at Kingsborough to the first two 
questions, and Chapter 5 provides some answers to the third. The impacts presented in this 
report cover up to two years following random assignment. MDRC will continue to track 
outcomes for the research sample for at least three years to understand the longer-term ef- 
fects of the Kingsborough program, including on labor market outcomes (the fourth ques- 
tion listed above), and will present the results in a future report. 

In each site in the study, MDRC and its research partners collected a range of data 
to help answer the impact and implementation research questions, including student tran- 




script data and a survey that was administered about a year after students entered the study. 
Chapter 2 provides more information on the data sources used in the analyses in this report. 
In addition, MDRC has collected data on the costs of operating each of the Opening Doors 
programs, as well as on the costs of providing each college’s standard classes and services. 
A future report will discuss the costs of the programs. 

A Limitation of the Research Design 

A random assignment evaluation is considered the most reliable way to test pro- 
grams such as those in Opening Doors, but this study’s design has an important limitation. 
The focus groups with students and the other research that MDRC conducted when design- 
ing Opening Doors pointed to the need for programs with multiple enhancements and sup- 
ports. Also, it was important to ensure that the program interventions were as robust as pos- 
sible and clearly different from the colleges’ standard offerings. As a result, as noted above, 
each college operated a program with at least two different components. The study will de- 
termine whether that package of enhancements led to different outcomes, compared with 
the standard college classes and services. The evaluation will not, however, disentangle the 
effects of each component. 

In Kingsborough, the study will determine whether the package of the learning 
community, enhanced counseling and tutoring, and textbook voucher lead to different stu- 
dent outcomes, compared with unlinked, standard courses and the college’s standard ser- 
vices. It will not, however, determine the specific effect of each of the program’s compo- 
nents, such as the enhanced counseling or textbook voucher. In addition, the study will not 
determine the effect of the difference in class size discussed in Chapter 3 or in any variation 
in quality of instruction. The implementation research at each college will shed light on 
which program dimensions may matter the most, but it will not yield definitive answers. 
That question will need to be addressed in future studies. 



Kingsborough Community College and Its Opening Doors 
Program 

This section describes Kingsborough and its setting and the origins of the college’s 
Opening Doors Learning Communities program. It also briefly describes the program 
model. The section begins by providing some background on learning communities, the 
central element of Kingsborough’ s program. 
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Some Background on Learning Communities 

Learning communities are seen by many as a promising strategy to promote student 
involvement with faculty, peers, and the subject matter they are studying, as well as student 
success. 8 Many experts believe that learning communities are particularly promising for 
students taking developmental (remedial) courses. 9 A key monograph on learning commu- 
nities offers the following definition: “any one of a variety of curricular structures that link 
together several existing courses — or actually restructure the material entirely — so that 
students have opportunities for deeper understanding and integration of the material they 
are learning, and more interaction with one another and their teachers as fellow participants 
in the learning enterprise.” 10 The four most common models of learning communities are 
paired or clustered courses, cohorts in large classes, team-taught programs, and residence- 
based programs. 11 The first model — upon which Kingsborough’s program is based — 
links two or more individually taught courses. From 20 to 30 students take the courses to- 
gether as a cohort, and, often, faculty redesign the curricula for the separate courses. Also, 
the courses are block scheduled, so that they meet Monday through Thursday, usually be- 
tween 9 A.M. and 2 P.M. 

The roots of the learning community approach can be traced back as far as the 
1930s, to an experimental program at the University of Wisconsin that redesigned the first 
two years of college around an interdisciplinary study of democracy in ancient Athens and 
modern-day America. 12 The learning community models that are more familiar today, such 
as those described above, began to spring up in the 1970s. By 2002, the National Survey of 
First- Year Academic Practices found that 62 percent of responding colleges enrolled at least 
some cohorts of students into two or more courses. However, at most colleges, these pro- 
grams involved only a small proportion of students. For example, about 60 percent of two- 
year colleges enrolled at least some students in learning communities, but fewer than 20 
percent of these colleges enrolled more than 10 percent of freshmen in such programs. 13 

Many community colleges adopt learning communities with the goal of improving 
the retention, persistence, and success of their most vulnerable students. Tinto and Eng- 
strom have conducted and synthesized research on student attrition and articulated the theo- 
retical underpinnings of learning communities. 14 They posit that increasing students’ at- 

8 The section on learning communities is adapted from Bloom and Sommo (2005); Price (2005); and 
Richburg-Hayes, Visher, and Bloom (2008). 

9 See, for example, Grubb (2001). 

10 Gabelnick, MacGregor, Matthews, and Smith (1990). 

1 'Levine Laufgraben and Shapiro (2004). 

12 Shapiro and Levine (1999). 

13 Barefoot (2002). 

14 Tinto (1993, 1997, 1998); Engstrom and Tinto (2007). 
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tachment to the college and increasing their engagement through academic and social ex- 
periences in college are important factors in improving persistence and retention. According 
to Tinto, building a sense of community, both academically and socially, is central to solv- 
ing attrition problems. 15 Astin studied both individual and organizational factors that were 
associated with persistence and found that engagement, particularly formed by interactions 
among students and between faculty and students, was important. 16 

An excellent resource that summarizes much of the research on learning communi- 
ties is from the National Learning Communities Project monograph series. 17 Many studies 
of learning communities have discussed the programs’ implementation and described stu- 
dents’ and instructors’ experiences. Some evidence suggests that learning communities can 
increase students’ integration and sense of belonging in the college community and their 
overall satisfaction with their college experience. For example, one study cited in the 
monograph found that learning communities helped students make connections across dis- 
ciplines and with peers. 18 Another recent, large-scale study of learning communities for 
academically underprepared students in 19 colleges found statistically significant impacts 
on engagement, the quality of relationships with classmates and faculty, and differences in 
classroom activities, such as participation in class discussions and using ideas from other 
courses. 19 

Despite the breadth of research on the learning community experience, few studies 
have measured the effect of learning communities on key student outcomes, such as persis- 
tence, course completion, and degree attainment. A few studies cited in the monograph 
mentioned above found that students in learning communities had the same grades as or 
better grades than similar students who were not in learning communities and were more 
likely to remain in college. 20 The study of learning communities for academically underpre- 
pared students in 19 colleges (see the preceding paragraph) found a modest but statistically 
significant positive effect on retention one year after enrolling in a learning community, 
averaged across the colleges. 21 

In sum, although past research on learning communities has found generally prom- 
ising results, few studies have employed rigorous research designs, and none of the large- 
scale studies have used a random assignment research design. The Opening Doors random 



15 Tinto ( 1993). 

16 Astin (1993). 

17 Taylor (2003). 

ls Russo (1995); cited in Taylor, Moore, MacGregor, and Lindblad (2003). 

19 Engstrom and Tinto (2007). 

2,l Tinto (1997); Tinto, Russo, and Kadel (1994); Gordon, Young, and Kalianov (2001). 
21 Engstrom and Tinto (2007). 
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assignment evaluation at Kingsborough provides an important contribution to the growing 
body of research on learning communities. 

Kingsborough Community College and Its Environment 

Before describing Kingsborough’ s Opening Doors Learning Communities program, 
it is worthwhile to consider the college and its environment. Kingsborough Community 
College is the only community college in Brooklyn, the largest borough of New York City, 
with a population of roughly 2.5 million people. Brooklyn has no majority racial/ethnic 
group: 43 percent are white; 35 percent are black; 9 percent are Asian; and 20 percent are 
Hispanic (any race). Close to half of all Brooklyn residents speak a language other than 
English at home. In 2006, 23 percent of Brooklynites lived below the federal poverty 
level. 22 This poverty rate is higher than the rate across the five boroughs of New York City, 
19 percent, and the national poverty rate, 13 percent. In 2006, the average unemployment 
rate in Brooklyn was 7.4 percent, a bit lower than the rate across the five boroughs of New 
York, 7.8 percent. The unemployment rate in the United States that year was 6.4 percent. 23 

Founded in 1963, Kingsborough is one of six community colleges in the City Uni- 
versity of New York (CUNY), the nation’s largest urban university system. 24 A large urban 
college, Kingsborough’ s 70-acre campus boasts an unexpectedly scenic location next to a 
quiet residential neighborhood at the southern tip of Brooklyn, on the Atlantic Ocean. The 
campus has numerous outdoor sculptures and an array of amenities, including a beach, an 
aquarium, and an art gallery. The college has more than 800 staff and an operating budget 
of nearly $60 million. 

Kingsborough serves about 35,000 students annually. The college reports that its 
students come from 110 countries and speak 68 languages. Kingsborough’ s student body is 
younger than that of many other community colleges — nearly three-fourths of Kingsbor- 
ough students are under age 25 — and a large fraction of students (about half) attend full 
time. The college’s Web site reports that Kingsborough ranks in the top 3 percent of com- 
munity colleges nationwide in the number of degrees awarded to minority students. 25 



22 The federal poverty level in 2006 for a family of four was $20,614 (U.S. Bureau of the Census, 
2006b). 

23 The statistics in this paragraph are from the 2006 American Community Survey (U.S. Bureau of the 
Census, 2006a). 

24 CUNY also includes 11 four-year institutions, a graduate school, a law school, and a school of 
biomedical education. 

25 Kingsborough Community College Web site: www.kbcc.cuny.edu. 
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Community colleges in New York State are among the most expensive in the na- 
tion: 26 Full-time Kingsborough students pay more than $3,000 per year in tuition and fees. 
At the same time, New York has an unusually generous state-funded financial aid program, 
the Tuition Assistance Program (TAP), which provided more than $800 million in grants to 
nearly 400,000 students in 2003. Nearly 70 percent of Kingsborough students receive some 
form of federal or state aid. 

The college, which accepts all students with a high school diploma or General Edu- 
cational Development (GED) certificate, provides developmental-level courses and English 
as a Second Language (ESL) courses to help prepare students for other college courses. 
Kingsborough offers a wide array of associate’s degree programs, including career-oriented 
programs and traditional liberal arts programs. Kingsborough’ s Office of Continuing Edu- 
cation operates the college’s certificate training programs, customized training for business, 
preparation for the GED certificate exam, and other noncredit programs, while the academic 
side of the college operates its associate’s degree programs. 

The distinction between “credit” and “noncredit” programs at Kingsborough is 
complicated by the fact that many students in the degree programs take noncredit develop- 
mental English and math courses because they fail one or more of the skills assessment tests 
that are administered to all students who enter the CUNY system. Developmental courses at 
Kingsborough do not earn any college credit, although each such course is assigned 
“equated credits” to account for the hours spent in class. Equated credits count in determin- 
ing whether a student is attending school full time. 

In 1998, the CUNY Board of Trustees, with the strong support of Mayor Rudolph 
Giuliani, voted to eliminate developmental (remedial) classes from the university’s four- 
year colleges. Once this policy took effect, students who failed any of the three skills as- 
sessment tests that are administered prior to enrollment — which covered reading, writing, 
and math — were directed to a community college for developmental coursework. The 
change generated great controversy. Critics, citing data showing that students who begin at 
community colleges are less likely to complete bachelor’ s degrees than comparable students 
who begin at four-year colleges, argued that the change would ultimately reduce the number 
of poor and minority students who obtained bachelor’s degrees. 27 

Although all incoming freshmen at Kingsborough have earned a high school di- 
ploma or GED, only a small minority pass all three skills assessment tests. In 2003, when 
the Opening Doors study started, only 18 percent of incoming students passed the reading, 



26 American Association of Community Colleges (2003). 
27 See, for example, Lavin and Weininger (1999). 
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writing, and math tests — a rate that is typical for many urban community colleges. Stu- 
dents who fail the tests are not required to take developmental courses when they start 
school, but students in all of Kingsborough’s associate’s degree programs must complete 
any required developmental English courses, plus two credit-bearing English courses that 
are open only to students who have passed both the reading and the writing skills tests. 
Similarly, students cannot transfer to a CUNY four-year college until they have passed all 
three of the skills tests. In general, students who fail the skills tests before enrollment can 
retake the tests only after they have passed specific developmental courses. 

In sum, the skills assessment tests at CUNY act as a “gatekeeper” to an associate’s 
degree or transfer to a four-year institution. In addition, as noted above, graduation rates for 
students who begin college in need of developmental education are particularly low. A key 
challenge and focus at Kingsborough, and in higher education more broadly, is how to ef- 
fectively help students complete developmental requirements and move to college-level 
courses. (As discussed below, most of the students in the Opening Doors study at Kings- 
borough took a developmental English course. A goal of the program was to help students 
move more quickly through developmental English requirements, allowing them to take 
college-level English.) 

Origins of the Opening Doors Learning Communities Program 

Kingsborough’s first learning communities program began in the mid-1990s and 
targeted English as a Second Language (ESL) students who were entering degree programs. 
Later, the college created another learning community program that targeted students in 
four “career majors”: accounting, business, mental health, and early childhood education. 
Data collected by the college showed that students in the learning communities had higher 
rates of course completion and semester-to-semester retention and higher grade point aver- 
ages than students at Kingsborough who were not in a learning community. 

Based on this positive experience, the leadership at Kingsborough was eager to ex- 
pand learning communities to a broader group of students. MDRC approached the college’s 
leadership in 2002 to discuss the possibility of Kingsborough’s participation in the Opening 
Doors demonstration, which offered an opportunity not only to expand learning communi- 
ties but also to rigorously test their effects on students. With special funding from the Robin 
Hood Foundation, which supports programs that aim to reduce poverty in New York City, 
Kingsborough developed the Opening Doors Learning Communities program and started 
operating it in fall 2003. 
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The Opening Doors Learning Communities Model 

Kingsborough targeted its Opening Doors Learning Communities program to stu- 
dents who met the following criteria: 

• Was a first-time incoming freshman who planned to attend college full time 
during the day 

• Did not test into ESL (that is, tested into either developmental English or col- 
lege-level English) 

• Was age 17 to 34 28 

ESL students were excluded from the study because they participated in the special 
learning communities program mentioned above. Similarly, students in the four “career ma- 
jors” for which a separate learning community operated were excluded for the first year of 
the study. The career learning community program ended after the 2003-2004 academic 
year, and, after that, students in those majors were included in Opening Doors. 

In designing Opening Doors Learning Communities, Kingsborough administrators 
were especially interested in targeting liberal arts majors, because they believed that many 
students in that group do not have clear academic or career goals and so might benefit from 
a model that provided enhanced structure and support. In addition, they made an effort to 
target students who missed the CUNY systemwide application deadline and applied directly 
to Kingsborough, often just weeks or even days before the start of classes. College data 
showed that these two overlapping groups of students tended to have poor outcomes, sug- 
gesting that they might benefit from Opening Doors. 

The program placed students in groups of up to 25 that took three classes together 
during their first semester at Kingsborough, forming a learning community. The paired or 
clustered courses consisted of an English course, usually at the developmental level; an 
academic course required for the student’s major; and a one -credit freshman orientation 
course. In addition to the linked courses, the Opening Doors program also offered addi- 
tional components designed to help students succeed: enhanced counseling and support, 
which was provided by the instructor of the orientation course; enhanced tutoring; and text- 
book vouchers for use at the college’s bookstore. Chapter 3 provides more detail on Kings- 



28 During the first semester of program operations, Kingsborough' s Opening Doors program was open 
only to students between ages 1 8 and 34 who reported household income below 250 percent of the federal 
poverty level. In subsequent semesters, the income criterion was removed, having been deemed unneces- 
sary because such a large proportion of Kingsborough students are from low- or moderate-income fami- 
lies, and 17-year-olds were admitted to the program with parental consent. 
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borough’s program model as well as the on the key differences between the program and 
the services that were available to the study’s control group. 



Contents of the Report 

Chapter 2 describes how students entered the research sample for the study at 
Kingsborough. It also presents some descriptive characteristics of the sample members and 
describes the data sources used in this report. 

Chapter 3 provides further information about Kingsborough’ s Opening Doors pro- 
gram and discusses its implementation. 

Chapter 4 presents the program’s effects on various educational outcomes. 

Chapter 5 provides information on the program’s effects on selected social, psycho- 
logical, and health outcomes. 
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Chapter 2 

Sample Intake and Characteristics and Data Sources 



The Opening Doors evaluation is using a random assignment research design to es- 
timate the effects of each college’s program, compared with their regular classes and ser- 
vices. This chapter describes how students became part of the research sample at Kingsbor- 
ough Community College and presents some characteristics of the sample members. It also 
discusses the data sources used in this report and the follow-up periods for the impact 
analyses. 



Identifying, Recruiting, and Randomly Assigning Students 

To enroll students in the study, MDRC and Kingsborough staff worked together to 
insert the recruitment and random assignment procedures into the college’s student registra- 
tion process. This was a considerable challenge, as large community colleges must process 
many student registrations in a short time period. In addition, this was the first time that a 
large-scale random assignment study had been conducted at a community college. The de- 
sign and research procedures were based on past studies set in other contexts, but they had 
to be adapted so that they would disrupt the normal college processes as little as possible. 
The Kingsborough administrators’ and staff members’ commitment to the study, coupled 
with their creativity and flexibility, were critical to the success of the research sample re- 
cruitment and intake processes. 

Potential study participants were identified during the weeks prior to the start of 
each semester. Kingsborough staff began by reviewing lists of applicants who had already 
taken the City University of New York (CUNY) skills assessment tests; as noted in Chapter 
1, scores on the reading and writing tests that are administered before enrollment determine 
most students’ English placements. Applicants whose scores placed them in a developmen- 
tal English course for native English speakers or in freshman English were invited to come 
to campus to register early for classes. 1 

Students who came in to register received a brief, general description of the Open- 
ing Doors program at Kingsborough and were told that the program had sufficient funding 



1 As discussed in Chapter 1, because some students were eligible for the English as a Second Language 
(ESL) learning communities program, those whose scores placed them in ESL were not included in Kingsbor- 
ough' s Opening Doors study. 
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to serve about half of eligible freshmen. Further, they were told that the program was part of 
a study, that it was open only to students who agreed to be in the study, and that a random 
process would be used to determine which study participants would be placed in the pro- 
gram. Students who agreed to participate in the study signed an informed consent form, 
provided some baseline demographic information, and completed a brief confidential sur- 
vey. They received a $20 transit card as an incentive and as compensation for their time, 
and they were then randomly assigned either to the program group or to the control group 
and were given appropriate assistance registering for classes. 2 

This sequence was ideal, but, in reality, most Kingsborough freshmen apply and 
take the CUNY skills assessment tests so close to the start of the semester that they were 
unable to attend an early-registration appointment. As a result, the majority of sample 
members entered the study during four or five large registration sessions that occurred in 
the few weeks before each semester began. Opening Doors Learning Communities and 
MDRC staff attended these sessions and “intercepted” freshmen who had just learned their 
test scores and were about to register for classes. Potential study participants heard the ex- 
planation about the research and the learning communities program and, if interested, com- 
pleted the research paperwork in small groups, rather than individually. Random assign- 
ment was conducted on the spot (typically through a phone call to MDRC’s office), and 
students proceeded to register for classes. Chapter 3 describes how registration occurred for 
program group and control group members. 

Students were brought into the research sample in four different groups, or cohorts, 
just before four different semesters: fall 2003, spring 2004, fall 2004, and spring 2005. 
Throughout the study, a total of 1,534 students were randomly assigned at Kingsborough. 



Characteristics of the Sample 

Table 2.1 presents some characteristics of the sample members at Kingsborough, 
based on the questionnaire that they completed just before random assignment: the Baseline 
Information Form (BIF). As the table shows, just over half of the sample members are fe- 
male. The research sample, like the population of Brooklyn, is racially and ethnically di- 



2 It is worth noting that students who came to a random assignment appointment and who were placed in 
the control group were — like the program group sUidents — allowed to register for classes earlier than most 
Kingsborough freshmen, and they received advice on the registration process from Opening Doors staff. These 
slightly enhanced services mean that the research design is not completely “pure,'’ but it was deemed unethical 
and impractical to bring students to campus and then not allow them to register for classes. 
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The Opening Doors Demonstration 
Table 2.1 

Selected Characteristics of Sample Members at Baseline 



Kingsborough Community College Report 



Characteristic 


Percentage 
of Full Sample 


Gender 


Male 


45.4 


Female 


54.6 


Age 


17-18 years old 


44.5 


19-20 years old 


34.2 


21-34 years old 


21.3 


Marital status 


Married 


3.9 


Unmarried 


96.1 


Race/ethnicity a 


Hispanic/Latino 


20.4 


Black, non-Hispanic 


37.7 


White, non-Hispanic 


26.9 


Asian or Pacific Islander 


8.6 


Other 


6.4 


One child or more in household 


8.7 


Household receiving any government benefits 11 


28.4 


Financially dependent on parents 


74.2 


Ever employed 


78.2 


Currently employed 


35.5 


Diplomas/degrees earned 


High school diploma 


70.9 


General Educational Development (GED) certificate 


28.6 


Occupational/technical certificate 


2.0 


Date of high school graduation/GED receipt 


During the past year 


70.2 


Between 1 and 5 years ago 


22.8 


More than 5 years ago 


7.0 


Main reason for enrolling in college 


To complete a certificate program 


2.8 


To obtain an associate's degree 


29.7 


To transfer to a 4-year college/university 


50.2 


To obtain/update job skills 


10.8 


Other 


8.4 



(continued) 
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Table 2.1 (continued) 



Characteristic 


Percentage 
of Full Sample 


First person in family to attend college 


33.4 


Working personal computer in home 


79.7 


Owns or has access to a working car 


25.6 


Language other than English spoken regularly in home 


46.9 


U.S. citizen 


72.6 


Respondent bom outside U.S. C 


39.9 


Respondent or respondent's parent(s) born outside U.S. C 


74.4 


Region in which respondent was born 


North America 


60.0 


Asia 


6.3 


Commonwealth of Independent States 


9.5 


Latin America and the Caribbean 


18.7 


Other e 


5.5 


f 

Region in which respondent's mother was born 


North America 


28.2 


Asia 


9.8 


Commonwealth of Independent States 


11.0 


Latin America and the Caribbean 


41.5 


Other 6 


9.6 


Sample size 


1,534 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 



NOTES: a Respondents who indicated that they are Hispanic and who also chose a race are included 
only in the Hispanic/Fatino category. 

b Benefits include unemployment/dislocated worker benefits. Supplemental Security Income (SSI) 
or disability, cash assistance or welfare, food stamps, and Section 8 or public housing. 

C "U.S." includes Puerto Rico. 

d This commonwealth comprises Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, 
Republic of Moldova, Russia, Tajikistan, Turkmenistan (until August 2005), Ukraine, and Uzbekistan. 

e Other regions include the Baltics, eastern and western Europe, North Africa, Sub-Saharan Africa, 
the Near East, and Oceania. Countries are grouped by region according to the U.S. Bureau of the 
Census, International Data Base. 

f The majority of respondents reported that both parents were born in the same region as each 

other. 



20 








verse: 38 percent of the sample members identified themselves as black (non-Hispanic); 27 
percent, as white (non-Hispanic); and 20 percent, as Hispanic. 

Reflecting the makeup of the college’s entering full-time freshmen, the sample 
members were quite young when they entered the study: 45 percent were either 17 or 18, 
and only 21 percent were 21 or older. Very few of the Kingsborough sample members were 
married or had children. Almost three-fourths reported being financially dependent on their 
parents, and roughly one-third were working when they entered the study. 

Most of the sample members (70 percent) had received their high school diploma or 
General Educational Development (GED) certificate during the past year. A small minority 
(7 percent) received their diploma or GED certificate more than five years before entering 
the study. Almost four-fifths of the sample members reported that their main reason for en- 
rolling in college was either to obtain an associate’s degree or to transfer to a four-year in- 
stitution. One-third of the students in the study said that they were the first in their family to 
attend college. 

Almost half of the sample members reported speaking a language other than Eng- 
lish at home — the same proportion as in Brooklyn overall. A full 40 percent of the students 
in the study were born outside the United States: 19 percent were born in Latin America or 
the Caribbean; 10 percent were born in what is now known as the Commonwealth of Inde- 
pendent States (a group of former republics in the Soviet Union); and 6 percent were born 
in Asia. Almost three-fourths of the sample members reported that either they or at least one 
of their parents were born outside the United States. 

Appendix Table A.l shows the complete list of characteristics that were collected 
on the BIF. The table shows the characteristics for the full sample, the program group, and 
the control group. An asterisk in the rightmost column of the table indicates that the propor- 
tion of program group members with that characteristic is significantly different than the 
proportion of control group members. As the table shows, there are some small differences 
between the two research groups, but no more than would be expected randomly. 3 

As noted above, students at Kingsborough are required to take CUNY skills as- 
sessment tests prior to beginning classes. As Table 2.2 shows, three-fourths of the study’s 
sample members passed the reading test, but only 29 percent passed the writing test, and 29 



3 Appendix Table A.l does not include an additional statistical test (an omnibus F-test, applied to evaluate 
the joint significance of the individual characteristics); it showed that there are no systematic differences be- 
tween the two research groups. 
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The Opening Doors Demonstration 
Table 2.2 



Outcomes on English Skills Assessment Tests at Baseline 
Kingsborough Community College Report 



Outcome 


Percentage 
of Full Sample 


Passed the reading test 


74.6 


Passed the writing test 


29.3 


Passed both the reading and the writing test 


29.0 


Sample size 


1,534 



SOURCE: MDRC calculations from City University of New York skills assessment test data. 



percent passed both tests. Thus, only a minority of the sample members could avoid devel- 
opmental-level English. 



Data Sources and Follow-Up Periods 

To examine the impacts and implementation of Kingsborough’ s Opening Doors 
program, the analyses presented in this report rely on several data sources, described below. 

Baseline Data 

As mentioned above, just before students were randomly assigned to the study 
groups, they completed a questionnaire, called the Baseline Information Form (BIF), and a 
baseline survey. The BIF collected demographic and other background information. The 
survey asked a series of questions about students’ social and psychological well-being and 
their health. Baseline data were used to describe the research sample, make statistical ad- 
justments in the impact analysis, and define subgroups of sample members for analysis. 

Kingsborough Transcript Data 

Kingsborough provided to MDRC transcript data for the sample members in the 
study. These data include various academic outcomes, including courses registered for, 
withdrawn from, and passed; number of credits earned; and grade point average (GPA). In 
this report, transcript outcomes are presented for the first semester that each sample member 
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was in the study (called the “program semester”) and for the subsequent three semesters 
(called “postprogram semesters”), yielding a follow-up period of two years. Transcript data 
are used in Chapter 4 to provide a detailed look at sample members' performance at Kings- 
borough. 

CUNY Skills Assessment Test Score Data 

As discussed above, students are required to take the CUNY reading, writing, and 
math skills assessment tests before they begin classes at Kingsborough. MDRC collected 
test score data for all sample members who took the tests at Kingsborough or any other in- 
stitution in the CUNY system. When the analyses for this report were conducted, test score 
data were available through the second postprogram semester. Test score data are used in 
this chapter and in Chapter 4. 

National Student Clearinghouse Data 

The National Student Clearinghouse, a nonprofit organization, collects and distrib- 
utes enrollment, degree, and certificate data from more than 3,000 colleges that enroll more 
than 90 percent of the nation’s college students. 4 The Clearinghouse data are used in Chap- 
ter 4 to provide important information about students in the study who may have attended a 
postsecondary institution other than Kingsborough. The Clearinghouse data provide infor- 
mation through the third postprogram semester, covering two years after sample members 
were randomly assigned. 

The Opening Doors 12-Month Survey 

A survey was administered to sample members approximately 12 months after ran- 
dom assignment. 5 MDRC attempted to locate and interview all sample members; in the end, 
7 1 percent of the full research sample at Kingsborough completed the survey. The survey 
asked about a wide range of topics, including sample members’ educational experiences, 
social relationships and supports, future outlook and identity, and health. Selected measures 
from the survey are used in Chapters 4 and 5. 



4 National Student Clearinghouse (2007). 

5 The Opening Doors 12-Month Survey used some questions from the Community College Survey of Stu- 
dent Engagement, with permission. 
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The Kingsborough Student Survey 

During fall 2004, MDRC surveyed program and control group members who had 
entered the study that semester. 6 The survey measured students’ experiences at Kingsbor- 
ough, including activities and experiences in their courses, relationships with other students, 
and awareness of student services. The survey was targeted to all 242 program group mem- 
bers enrolled in an Opening Doors learning community that semester and to a random sub- 
set of 138 control group members; 84 percent of the program group members and 73 per- 
cent of the control group members responded. Because the sample size is much smaller than 
the sample for the Opening Doors 12-Month Survey, and because the Kingsborough Stu- 
dent Survey is limited to only one of the four research cohorts, the data are used to illustrate 
some important information about program implementation in Chapter 3, but the report’s 
impact analyses do not rely on the Kingsborough survey data. 

The Kingsborough Faculty Survey 

In early 2005, MDRC distributed a survey to the faculty members who taught in the 
Opening Doors Learning Communities program during the fall 2004 semester. The survey 
asked a series of questions about the faculty members’ experiences teaching in the learning 
community and covered such topics as interactions with students and other faculty members 
and the curricula and instruction in the learning community courses. Of the 24 faculty 
teaching in an Opening Doors Learning Community that semester, 2 1 completed the survey. 
Information from the faculty survey is used in Chapter 3 to help describe the implementa- 
tion of the Opening Doors program at Kingsborough. 

Field Research 

During spring 2005, MDRC staff conducted a series of field research visits to 
Kingsborough. MDRC interviewed many college administrators, faculty, and staff, includ- 
ing those involved in Opening Doors. The interviews provided detailed information on the 
operation of the Opening Doors Learning Communities program and about the key differ- 
ences between the program and the standard college courses and services that were avail- 
able to the members of the study’s control group. Throughout the study, prior to the formal 
field research visits, MDRC periodically interviewed the administrators and staff involved 
in Opening Doors and observed meetings between counselors and students. Information 
from all these visits is used in Chapter 3. 



6 The Kingsborough Student Survey was adapted from the Pathways to Success Survey, created by Vin- 
cent Tinto. The Pathways to Success Survey is a modification of the Community College Survey of Student 
Engagement and was used with permission. 
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The Opening Doors Qualitative Study 

MDRC conducted an in-depth qualitative study at Kingsborough and another Open- 
ing Doors college, Lorain County Community College in Elyria, Ohio, outside Cleveland. 
During spring 2005, interviews were conducted with 23 students from the program and con- 
trol groups at Kingsborough to learn about their experiences in college and the factors that 
affected their ability to stay in school. The study’s results were presented in a report in 
2006, 7 and some of the interview data are used in Chapter 3. 



7 Gardenhire-Crooks, Collado, and Ray (2006). 
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Chapter 3 



The Implementation of 
Opening Doors Learning Communities 



The chapter describes how the Opening Doors Learning Communities program operated 
at Kingsborough Community College during the study period. After a brief summary, the chapter 
describes the program model and stmcture, examines how Kingsborough administrators mounted 
the program in fall 2003, discusses key operational issues that emerged over time, and finally de- 
scribes how Opening Doors Learning Communities differed from the regular college environment 
facing the study’s control group. 

The key findings are: 

• Opening Doors Learning Communities was well implemented at Kingsbor- 
ough. Despite a compressed planning period and the program’s large scale, 
all the key features were in place when operations began, and they remained 
in place throughout the study period. In interviews, many faculty, students, 
and administrators expressed positive views about the program. These ac- 
complishments were possible because the program received strong, consis- 
tent support from the highest levels of the college administration. 

• All the learning communities had the same basic structure, but they varied in 
their content and class sizes and in the degree to which faculty worked to- 
gether and integrated their courses. Thus, while this study is a strong test of 
the structural features of a learning community (linked courses, block sched- 
uling, and so on), it may not fully test the impacts of tightly integrating 
course curricula. That said, it seems likely that Kingsborough’ s program is at 
least as strong as, if not stronger than, the “typical” learning communities 
program at a community college, and so the results are probably widely ap- 
plicable. 

Finally, it is important to note that Kingsborough is, in general, a high-functioning com- 
munity college. Thus, the study’s control group probably received relatively high-quality instruc- 
tion and relatively strong support. This means that the benchmark for achieving impacts may be 
higher at Kingsborough than it might have been at another college. 
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The Opening Doors Learning Communities Model 

The Opening Doors Learning Communities program began in fall 2003 and still operates 
today. However, members of the Opening Doors research sample experienced the program only 
during its first four semesters of operation, from fall 2003 through spring 2005 (known as the 
“study period”). After the study period ended, Kingsborough expanded the program, with the goal 
of serving 80 percent of entering freshmen by 2010, and it made some modest changes in the 
model. At the time this report was written, about 65 percent of entering freshmen participated in 
Opening Doors Learning Communities. This chapter reflects the model and operations only dur- 
ing the study period (and thus is written in the past tense). 

Program Components 

Opening Doors Learning Communities used the common paired- or clustered-course 
model described in Chapter 1 . That is, the students in each learning community took three courses 
together: an English course, with the level determined by the student’s scores on the City Univer- 
sity of New York (CUNY) reading and writing skills assessment tests that were administered prior 
to enrollment; an academic course required for their major, called a “content course” (for exam- 
ple, psychology, health, or history); and a one-credit freshman orientation class. This class, which 
is open to all Kingsborough freshmen, teaches time management, study skills, college rules and 
procedures, exploration of learning styles, career exploration, multicultural diversity, and other 
topics relevant to a new college student; similar courses are offered at many two-year and four- 
year colleges. As discussed later in the chapter, 3 1 of the 40 learning communities that operated 
during the study period included a developmental English course; the other 9 included a credit- 
bearing freshman English course for students who passed both the reading and the writing skills 
assessment tests prior to enrollment. 

Figure 3.1 shows the actual class schedules for 2 of the 1 1 Opening Doors learning com- 
munities that operated during fall 2004, the third semester of the study period. The first panel of 
the figure shows the schedule for a learning community that linked a credit-bearing (that is, non- 
developmental) English course (English 12) with a sociology course (Sociology 31) and the 
freshman orientation course (Student Development [SD] 10). The second panel shows the sched- 
ule for a learning community that linked the lowest-level developmental English course (English 
91) with a health course that is required for most Kingsborough students (Health 12) and the 
freshman orientation class. 
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The Opening Doors Demonstration 
Figure 3.1 

Schedules Showing Linked Courses in the 
2004 Opening Doors Learning Communities Program 

Kingsborough Community College Report 



Sociology Link (10) 

English 12 (ENG 12) and Sociology 31 (SOC 31) 



Period 


Monday 


Tuesday 


Wednesday 


Thursday 


Friday 


A 

(8:00-9:00) 












B 

(9:10-10:10) 












C 

(10:20-11:20) 












D 

(11:30-12:30) 


ENG 000LB 

(0.0 Credits) 










E 

(12:40-1:40) 


SOC 31 

(3.0 Credits) 


SD 10 

(1.0 Credit) 


SOC 31 

(3.0 Credits) 


SOC 31 

(3.0 Credits) 




F 

(1:50-2:50) 


ENG 12 

(4.0 Credits) 


ENG 12 

(4.0 Credits) 


ENG 12 

(4.0 Credits) 


ENG 12 

(4.0 Credits) 




G 

(3:00-4:40) 













(continued) 



As Figure 3.1 shows, the Opening Doors courses met one after the other (sometimes with 
a break for lunch). This scheduling system was designed to minimize the amount of time that a 
student needed to be on campus, making it easier to balance school with work and family obliga- 
tions. In addition, all Opening Doors courses met at convenient times, generally between 9 A.M. 
and 3 P.M., Monday through Thursday. 1 



'After the study period for this report ended, Kingsborough began offering learning communities for eve- 
ning students as well. 
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Figure 3.1 (continued) 



Health Link (1) 

English 91 (ENG 91) and Health 12 (HPE 12) 



Period 


Monday 


Tuesday 


Wednesday 


Thursday 


Friday 


A 

(8:00-9:00) 












B 

(9:10-10:10) 












C 

(10:20-11:20) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 




D 

(11:30-12:30) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 

(8.0 EQ Credits) 




E 

(12:40-1:40) 












F 

(1:50-2:50) 


HPE 12 

(3.0 Credits) 


HPE 12 

(3.0 Credits) 


HPE 12 

(3.0 Credits) 


SD 10 

(1.0 Credit) 




G 

(3:00-4:30) 













SOURCE: Kingsborough Community College. 



NOTES: The schedules represented here are actual schedules from Kingsborough Community College. 
Linked courses form the core of the Opening Doors Learning Communities program at Kingsborough. 
All combinations of Kingsborough's linked courses include one English course; one "content course" 
(covering an academic subject other than English); and Student Development 10 (SD 10), a skills 
course for new students. 

"EQ credits" stands for equated credits. Equated credits are weekly class hours in developmental 
and compensatory courses for which actual credit is not allowed. For certain purposes, such as 
determining financial aid eligibility, equated credits may be counted in the same manner as regular 
credits. 



A typical full-time course load at Kingsborough involves 12 credits (12 hours of class per 
week). Because the lower-level developmental English courses meet for eight hours each week, 
the content courses are typically three credits, and the freshman orientation class is one credit, stu- 
dents at the lower English levels usually took no additional unlinked courses. In contrast, students 
in higher-level English courses, which meet for fewer hours per week, usually took at least one 
non-Opening Doors course. For example, the linked Opening Doors courses for the schedule 
shown in the first panel of Figure 3. 1 total only eight credits. 
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Faculty who taught the linked courses had a reduced teaching load, allowing them to meet 
regularly during the semester to discuss student progress, identify strategies to assist students hav- 
ing difficulty, and coordinate assignments. In effect, a three-credit course taught in the context of 
Opening Doors was treated as a four-credit course when determining a faculty member' s teaching 
load. (After the first semester, participating faculty also received compensation for time spent 
planning their Opening Doors course during the six-week module preceding the semester.) 

In addition to the linked-course structure, the Opening Doors Learning Communities pro- 
gram included several other components designed to address students’ barriers to retention and 
academic success: 

• Enhanced counseling and support. In each learning community, the fresh- 
man orientation course was taught by an Opening Doors counselor (usually 
called a “case manager”), who worked proactively to identify and resolve 
students' barriers to good attendance and performance. Ideally, the case man- 
ager/instructor participated in regular meetings with the other two faculty 
members in a given learning community during the semester, creating an ef- 
fective “early-warning” system to identify students needing assistance — for 
example, students who had been missing classes or who were having diffi- 
culty with assignments. Typically, each Opening Doors case manager was 
responsible for three or four learning communities (75 to 100 students in all). 

• Enhanced tutoring. Normally, tutors are assigned to developmental English 
courses at Kingsborough — and may actually attend the classes — but oth- 
erwise students access tutoring by visiting a central lab. In the Opening 
Doors Learning Communities program, a tutor was assigned to each learning 
community and attended the English course and, in many cases, the subject- 
matter course as well. The aim was to ensure that tutors were familiar with 
the material being covered — and with the students. They were well posi- 
tioned both to help with the work in a given course and to help students draw 
connections across the linked courses. 

• Textbook vouchers. The high cost of college textbooks has been well 
documented, and studies have shown that many community college students 
do not purchase the books for their courses; they attempt to share or borrow 
books or simply get by without them. 2 Redeemable at the campus bookstore, 
the Opening Doors textbook voucher was worth up to $150 during the initial 



2 Community College Week ( 2003 ). 
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12-week session. Students who returned for the 6-week winter or summer 
module (described next) could receive a second voucher worth up to $75. 3 

At Kingsborough, each semester is divided into two “modules.” The primary module (fall 
or spring) is 12 weeks, but students who attend that module full time can earn additional credits by 
attending the subsequent 6-week module (winter or summer) for no additional cost. (Courses typi- 
cally meet for twice as many hours per week during the 6-week module.) Opening Doors Learn- 
ing Communities operated only during a student’ s first semester at Kingsborough. In fact, the core 
feature of the program — the linked classes — existed only during the first 12-week module. Stu- 
dents were no longer scheduled as a cohort during the subsequent 6-week module, although those 
who chose to attend could receive a second textbook voucher and were still assigned to their 
Opening Doors case manager. Also, as discussed below, there were some social events designed 
to help students transition out of Opening Doors and into the regular college environment. 

The Administrative Structure 

A well-known technical assistance guide on learning communities describes the sharp 
split between the academic and student affairs divisions on many college campuses, and it identi- 
fies this split as a serious barrier that must be overcome in order to run an effective learning com- 
munity program. 4 

The Opening Doors Learning Communities program grew out of an unusually close col- 
laboration between the academic affairs and student services divisions at Kingsborough. Senior 
administrators in both divisions had close personal relationships and a long history of working 
together. The Director of Academic Affairs led the program and was responsible for recruiting 
faculty, building the linked schedules, and other key tasks, but she worked very closely with the 
Director of the Freshman Year Experience and other staff in the student services division to design 
and oversee key features of the program. The complex student recruitment and registration proc- 
ess that is described in Chapter 2 was developed collaboratively by staff from the two divisions 
and MDRC. 

The close collaboration was reflected in the Opening Doors Learning Communities of- 
fice, which housed both an academic coordinator (part of the academic affairs division) and the 
case managers (part of the student services division). 



3 During the 2003-2004 academic year, the textbook vouchers were worth up to $200 during the 12-week 
session and up to $100 during the 6- week session. 

4 Shapiro and Levine (1999). 
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Program Start-Up 

In mounting Opening Doors, Kingsborough could build on the experience gained from its 
two earlier (albeit smaller) learning community initiatives described in Chapter 1 . (One of the ear- 
lier learning communities was for English as a Second Language students, and one was for stu- 
dents in four “career majors.”) At the same time, Kingsborough faced two special challenges. 
First, as discussed further below, operating the Opening Doors program in the context of a random 
assignment experiment created unique problems, particularly during the registration process. 

Second, to meet the overall project schedule and take full advantage of external funding, 
Kingsborough needed to mount the program, at a large scale, very quickly. Detailed planning did 
not begin until the spring of 2003, after course assignments and schedules had already been set for 
the fall semester. Compounding the challenge, the academic affairs administrator with lead re- 
sponsibility for planning the program became ill and missed several months of work just before 
and after the program was launched. One administrator noted that Kingsborough had to “build the 
ship while sailing it.” 

Given these difficult circumstances, it is impressive that Kingsborough was able to put the 
entire complex program structure in place for the start of the fall 2003 semester: Courses were 
identified and set aside for Opening Doors students; 10 block schedules were created; faculty were 
recruited to teach the Opening Doors courses; a mechanism for dispersing textbook vouchers was 
established; and the Opening Doors counseling staff were h ir ed and on board in time for registra- 
tion. This considerable achievement was possible only because Opening Doors received very 
strong support at all levels of the college administration and from many of the academic depart- 
ments. 



One of the most challenging aspects of the start-up process involved recruiting faculty to 
teach the linked courses. The technical assistance guide on learning communities that is men- 
tioned above notes that, while college faculty are often innovators in the classroom and in their 
academic disciplines, they are “frequently slow to accept changes that appear to alter traditional 
relationships between faculty and administration, faculty and students, faculty in their own de- 
partments, other departments, and the [college] community at large. In this respect, learning 
communities introduce serious challenges to the usual way of doing things.” 5 

The guide goes on to recommend a careful process for attracting and recruiting faculty to 
teach in learning communities. For example, it suggests giving faculty substantive roles in plan- 
ning the program, thereby creating an atmosphere that attracts other faculty. Unfortunately, given 
the extremely tight time frame for program start-up, such a process was not feasible at Kingsbor- 



5 Shapiro and Levine (1999). 
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ough. Thus, some of the faculty who agreed to teach in Opening Doors Learning Communities 
did so enthusiastically (some had participated in one of the college’ s earlier learning community 
initiatives). Others, however, were recmited by department chairs, did not have prior experience 
teaching in a learning community, and may not have fully understood what was involved. 

Similarly, the compressed schedule made it difficult to carefully consider how to pair fac- 
ulty to teach the linked courses. Ideally, faculty pairs would have emerged through an organic 
process, based on professional relationships or shared academic interests. In reality, during the 
first semester, pairings were created mostly based on schedules and other logistical factors, and 
many faculty were asked to collaborate with people they had never met. 

Finally, there was little time available to provide special training. Tasks such as integrat- 
ing curricula and working collaboratively with counselors or case managers were new to many of 
the participating faculty, and, for the most part, they needed to learn such tasks through trial and 
error. Adding to the challenge was the fact that faculty did not receive special compensation for 
planning efforts prior to the start of the first semester (that is, during the preceding six-week mod- 
ule). 



In interviews conducted near the end of the study period, Kingsborough administrators 
acknowledged that the initial faculty recruitment process was less than ideal owing to the tight 
time frame. As a result, they believed that Opening Doors may have been seen by some faculty as 
an administration initiative and that some of the initial faculty assignments were probably not ap- 
propriate. However, many of these issues were addressed in later semesters. For example, as 
knowledge about Opening Doors spread among faculty, more instructors began to come forward 
voluntarily to express interest in the program, and a waiting list eventually was needed. (When 
Kingsborough h ir ed a large number of new faculty in 2004, administrators asked all candidates 
about their interest in teaching in a learning community.) In some cases, pairs of faculty from dif- 
ferent departments proposed a specific linkage between courses they taught. In addition, faculty 
development opportunities (some of which were delivered by consultants associated with the Na- 
tional Learning Communities Project at Evergreen State College) were expanded in later semes- 
ters, and compensation for pre-semester planning was added. 

In a survey of faculty who taught in Opening Doors Learning Communities during the 
fall 2004 — the third semester of program operations — most of the faculty who wrote comments 
expressed quite positive views about the program. For example, four faculty, all from different 
learning communities, wrote the following comments: 

It is a very enjoyable, though time-consuming experience, but seeing positive 
results is worth the investment. 
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The program was awesome to teach in. Administrative support and student 
engagement were completely different (better) than non-OD courses. I asked 
to do it again. 

If there is a way to help students overcome their handicaps, the OD program 
is the way. Students become energized and motivated (for the most part). I 
believe Opening Doors is the way to conduct all classes. 

The Opening Doors program is a fabulous program for our KCC students. It 
provides them with an initial connection to faculty, as well as their peers. 

This study did not gather data on whether the faculty who taught in Opening Doors 
Learning Communities differed from other faculty at Kingsborough — though this is an important 
factor to consider in assessing the results and the potential for replication or expansion of such 
programs. 



Implementing Opening Doors Learning Communities 

The key structural features of the Opening Doors Learning Communities program at 
Kingsborough Community College operated as intended, with minor glitches, throughout the 
study period. Thus, the overwhelming majority of students were assigned to linked classes with a 
cohort of fellow students in their English level; tutors and case managers with relatively small 
caseloads were assigned to each learning community; and textbook vouchers were distributed as 
planned. In other words, the program as a whole was well implemented. 

This section goes beyond the basic structural elements of Kingsborough’ s program to dis- 
cuss the day-to-day operation of some of its key features: registering students, linking courses, 
providing enhanced support, and transitioning students out of the program. 

Registering Students 

Chapter 2 describes the process of recruiting students and randomly assigning them to the 
program group or the control group. Once students learned that they had been assigned to the pro- 
gram group, they met immediately with a case manager for advisement and registration. In many 
cases, this was a relatively straightforward process because the choices were quite constrained. 
Students' English placements were determined by their scores on the reading and writing skills 
assessment tests administered prior to enrollment. The Opening Doors Learning Communities 
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program served students who had been placed in one of the three noncredit developmental Eng- 
lish courses or the one credit-bearing freshman English course. 6 

As shown in Table 3.1, in fall 2004, students whose test scores placed them in the mid- 
level developmental English course (English 92) could choose from two possible schedules, but 
both included the same courses. Limited flexibility was inevitable, in part because there are rela- 
tively few college-level content courses that are appropriate for students whose reading and writ- 
ing skills are poor. 7 Students in that semester who tested into the highest level of developmental 
English (English 93) could choose from four different learning communities, each with a different 
content course. 

Table 3.1 also illustrates another challenge for the registration process: In order to design 
the learning communities, administrators needed to predict how many study participants would 
test into each of the four English levels and needed to create an appropriate number of learning 
communities at each level. They were guided by historical test score data, but the patterns tended 
to fluctuate from year to year. Moreover, administrators had no way to predict what proportion of 
the eligible students would agree to participate in the study. 

To make matters worse, random assignment greatly reduced the margin for error. In dis- 
cussing the random assignment process with MDRC up front, Kingsborough administrators 
wanted to fill the available Opening Doors slots and then assign any additional eligible students to 
the control group. Unfortunately, however, this would not have created comparable groups, be- 
cause the students who register earlier may be systematically different from those who register 
later. To make the process work for the evaluation, each student who entered the study had to 
have a 50 percent chance of being placed in the program group. 

As shown in Table 3.1, there were certain courses in each semester that were significantly 
underenrolled. For example, in fall 2003, the number of students who tested into the middle level 
of developmental English (English 92) was smaller than projected, so the three learning communi- 
ties including that course were quite small. The problem was more widespread in spring 2004, 
when the overall number of incoming freshmen eligible for Opening Doors Learning Communi- 
ties was much smaller than projected. Thus, the size of the learning communities varied substan- 



6 English 91 and 92 are lower-level developmental courses, targeted mainly to students who fail both the 
reading and the writing skills test. Both courses meet for eight hours per week. English 93, the highest-level 
developmental course (targeted to students who fail only the writing skills test) and English 12 (freshman Eng- 
lish) meet for four hours per week. 

7 The Health class that is linked to English 91 and English 92 is a schoolwide requirement. 
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The Opening Doors Demonstration 
Table 3.1 

Opening Doors Learning Communities Offered and Class Size 



Kingsborough Community College Report 



Semester and Linked Courses 


Number of 
Sections 


Average 
Class Size 


Fall 2003 






English 12, Philosophy, Student Development 10 


1 


20 


English 12, Sociology, Student Development 10 


1 


23 


English 91, Speech, Student Development 10 


2 


17 


English 92, Music, Student Development 10 


1 


6 


English 92, Health, Student Development 10 


2 


10 


English 93, Political Science, Student Development 10 


1 


13 


English 93, History, Student Development 10 


1 


19 


English 93, Psychology, Student Development 10 


1 


20 


Semester total 


10 


16 


Spring 2004 






English 12, Political Science, Student Development 10 


1 


10 


English 12, Sociology, Student Development 10 


1 


22 


English 91, Health, Student Development 10 


1 


16 


English 91, Speech, Student Development 10 


1 


7 


English 92, Health, Student Development 10 


2 


7 


English 93, Political Science, Student Development 10 


1 


12 


English 93, History, Student Development 10 


1 


10 


English 93, Psychology, Student Development 10 


1 


19 


Semester total 


9 


12 


Fall 2004 






English 12, Psychology, Student Development 10 


1 


25 


English 12, Sociology, Student Development 10 


1 


25 


English 12, Economics, Student Development 10 


1 


25 


English 91, Health, Student Development 10 


2 


25 


English 92, Health, Student Development 10 


2 


22 


(continued) 
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Table 3.1 (continued) 



Semester and Linked Courses 


Number of 
Sections 


Average 
Class Size 


Fall 2004 (continued) 






English 93, Political Science, Student Development 10 


1 


22 


English 93, Psychology, Student Development 10 


1 


21 


English 93, History, Student Development 10 


1 


24 


English 93, Biology, Student Development 10 


1 


9 


Semester total 


11 


22 


Spring 2005 






English 12, Sociology, Student Development 10 


1 


25 


English 12, Economics, Student Development 10 


1 


22 


English 91, Health, Student Development 10 


2 


17 


English 92, Health, Student Development 10 


2 


11 


English 93, Political Science, Student Development 10 


1 


14 


English 93, History, Student Development 10 


1 


13 


English 93, Psychology, Student Development 10 


1 


22 


English 93, Biology, Student Development 10 


1 


8 


Semester total 


10 


16 


Grand total 


40 


17 



SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTES: English 12 is the first-level college-credit English course. English 91, 92, and 93 are the 
lowest, middle, and highest levels, respectively, of developmental English courses. Enrollment in 
these courses is based on the score from a skills assessment test that is required of all entering 
students. Class size may not represent the full class size in every case, as students outside the study 
may have enrolled in a study learning community. However, such occurrences were rare and would 
not significantly change the numbers. 

While most students were enrolled in all three linked courses of a learning community, a small 
number withdrew from part of the link. 



tially, and most of the learning communities operated with many fewer than 25 students. Adminis- 
trators were reluctant to cancel or consolidate course sections after they had made commitments to 
faculty who had agreed to teach in the program and had already worked with their partner to re- 
design their course syllabus (see below). The process worked much more smoothly in the 2004- 
2005 academic year, as administrators and MDRC gained experience and worked together to re- 
fine the registration procedures. 
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Linking Courses 

The linked-course structure was the heart of the Opening Doors Learning Communities 
program. The structure was designed to achieve many goals: to help students build close, suppor- 
tive relationships with their peers to ease the transition into college; to enhance learning by em- 
phasizing the substantive linkages across different disciplines; and to facilitate closer connections 
among students, faculty, and case managers. As noted earlier, the structural elements of the linked 
structure were put in place quite effectively at Kingsborough. This section explores several aspects 
of the linked structure as it played out in practice: curricular links, ongoing communication among 
faculty teams, and social linkages among students. 

Curricular Links 

Designing courses to help students see substantive links across different subjects is a chal- 
lenging task, and, as noted above, many of the faculty who taught in Opening Doors Learning 
Communities did not have previous experience doing so. This was a major topic of the faculty 
development efforts, and while MDRC does not have detailed information on all 40 learning 
communities, it appears that faculty grew more comfortable with the task over time. 

In some learning community programs, courses are fully integrated under a single theme. 
At the other extreme, courses may be block-scheduled, with little integration. In the Kingsborough 
program, the two courses remained separate and distinct, but they were coordinated to varying 
degrees. 



As noted in Chapter 2, MDRC distributed a survey to Kingsborough faculty who taught 
Opening Doors Learning Communities during the fall 2004 semester, the third semester of pro- 
gram operations. There were 11 learning communities that semester, and 21 of the 24 participat- 
ing faculty completed the survey. 8 All faculty reported that they had developed a new syllabus or 
adapted their regular syllabus for the learning community. (It is not clear whether any of the learn- 
ing communities had a single syllabus.) All also reported that they gave at least some joint as- 
signments with their partner, and most reported that they developed a grading scheme together. 
The survey was not detailed enough to determine how often these strategies were used, but exam- 
ples of joint assignments or activities include the following: 

• Several English instructors reported that they assigned novels or other read- 
ings that related to the subject matter of the content course; several teams as- 
signed some of the same texts for both courses. 



8 There were 11 English instructors, 10 content-course instructors (1 instructor taught in two learning 
communities), and 3 freshman orientation instructors/case managers. 
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• Several teams made joint writing assignments that were graded or reviewed 
by both instructors. In other cases, English writing assignments included the 
requirement to use key terms from the content course. 

• In a learning community that linked health with English, the students learned 
about stress in the health class, wrote scripts about the topic in the English 
course, and acted out the scripts in health class with the English instructor 
present. 

• In a learning community that linked biology with English, the biology text 
was used in the English course, and a scientific paper was assigned for both 
classes and was graded by both instructors. 

• In one learning community, the English and content faculty team-taught two 
class sessions. 

From interviews with students conducted as part of a qualitative study (see Chapter 2), it 
seems clear that many Opening Doors Learning Communities participants were aware of, and 
appreciated, the linkages between their English and content course. One student who was inter- 
viewed for MDRC’s qualitative study noted: “Every time we read stories [for the English class], 
we somehow related it to our economics class. So ... I think it was kind of important.” Another 
said: “It doesn’t feel like you have different classes. Its like its all one class but different subjects. 
You can study easier. Use what you learned here [points to another place] here. Its like a web. Its 
all connected.” 9 Some students also believed that Opening Doors Learning Communities involved 
less work because the same assignments could sometimes be completed for two classes. 

Ongoing Communication Within Teams 

Ideally, all three members of each learning community team (the English instructor, the 
content-course instmctor, and the freshman orientation instructor/case manager) would meet prior 
to the semester to plan and integrate their courses, and then they would meet weekly or biweekly 
throughout the semester to review student progress, develop strategies to assist students in need, 
coordinate assignments and grading, and make midcourse corrections. 

When this system worked smoothly, faculty reported that it was quite helpful. However, 
interviews with Kingsborough faculty throughout the study period, and the survey of the fall 2004 
faculty, suggest great variation in how the teams functioned in practice. Although some of the re- 
ports by teammates are not consistent, results from the survey suggest that regular in-person meet- 



9 Gardenhire-Crooks, Collado, and Ray (2006). 
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ings including all three team members occurred in about 5 of the 1 1 learning communities that 
operated that semester. It appears that the English and content-course instructors met regularly, 
without the case manager/instructor, in at least four other links. 

Interviews with faculty suggest that the lack of ongoing communication could sometimes 
be traced to logistical and scheduling issues. At Kingsborough, as at most community colleges, a 
large percentage of the faculty (about two-thirds) are part-time adjuncts. Similarly, several of the 
Opening Doors Learning Communities case managers worked part time. When two or more 
members of a team had different part-time schedules, it could be very difficult to organize regular 
face-to-face meetings. Some of the teams that did not meet frequently reported that they commu- 
nicated by e-mail. 

It also seems clear that some faculty did not fully understand or appreciate what the case 
manager could contribute. As noted above, the academic affairs and student development divi- 
sions forged a very close linkage at the administrative level, but it was challenging to translate this 
message of collaboration to the faculty involved. It is notable that, in several of the learning com- 
munities that did not meet regularly as a full team, there was still somewhat regular contact be- 
tween the case manager/instructor and the English instmctor. It appears that the English depart- 
ment was more fully engaged in this aspect of the program than were the other departments — 
perhaps this is not surprising, because the English department contributed an instructor to every 
learning community. 

Social Linkages Among Students 

When discussing Opening Doors Learning Communities, students at Kingsborough were 
most likely to refer to the fact that they took most or all of their courses with the same people; this 
was the most visible aspect of the program to them. 

It is difficult to generalize based on the relatively small number of students who were in- 
terviewed, but it appears that most students appreciated this aspect of the program. They noted 
that they had made friends, that their classmates were supportive, and that they were more willing 
to participate in class discussions because they knew everyone in the class. Some students also 
reported that they formed study groups with their classmates. One student who was interviewed 
for the qualitative study said: “I make friends with people in my class, [and] if I’m absent and I 
need something, they’re there.” Another student, who was interviewed after leaving Opening 
Doors, contrasted the program to the regular school environment: “And now, I go to my classes 
and no one even takes me on! I hardly know anybody, like, if I need to call somebody for some- 
thing, I don’t know anybody to call, because I have no friends [in class]. It’s just me alone in these 
big classes.” A third student said: 
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Opening Doors was fun! We went to our classes and we did well. ... I had a 
friend . . . and we got so close through the OD semester that we would com- 
pete among ourselves to see who would get the highest grade in our English 
class, you know? And sometimes she would and sometimes I would, but if I 
got the lower grade, she would be, like, “Come on — you know you have to 
beat me next time!” 

At the same time, the authors of the qualitative study noted that most of the students said 
that the friendships they made in Opening Doors generally did not last beyond the program se- 
mester and that their closest friends were not from Kingsborough. In addition, some students dis- 
cussed negative aspects of the group structure, saying that they found it stifling and that it re- 
minded them of high school. 

Interestingly, many students expressed contradictory views about this aspect of the pro- 
gram, appreciating the ability to make friends and gain support but also feeling as though they 
were not being treated as adults. The authors of the qualitative study concluded that “this con- 
flicted yearning for both dependence and independence likely reflected their stage in life, at transi- 
tion point from adolescence to adulthood .” 10 In other interviews conducted by MDRC, students 
who were in the program complained that they were tired of attending classes with the same stu- 
dents every day. However, after leaving the program and experiencing the relatively anonymous 
life of a regular student, the same students admitted that they missed the program and understood 
the advantage of the group structure. 

Although many faculty expressed positive views about the group structure, several re- 
ported that it sometimes contributed to a negative dynamic that was hard to control. For example, 
in the faculty survey mentioned above, one instructor wrote: “As the semester goes on, students 
tend to form negative attitudes, with the naysayers holding others back. It’s a high schoolish clique 
kind of thing.” Another wrote: “Students without much maturity go from class to class with each 
other only, and the insularity can reinforce the immaturity and counterproductive behavior pat- 
terns.” 



Enhancing Student Services 

The enhanced student services component of Opening Doors Learning Communities re- 
volved around the case managers. The staffing structure at Kingsborough varied somewhat from 
semester to semester, but typically it included two full-time positions and one part-time staff per- 
son. One of the full-time staff was the lead case manager and played a broader coordination or 



10 Gardenhire-Crooks, Collado, and Ray (2006). 
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management role, in addition to teaching three sections of the freshman orientation class. The 
part-time staff person taught three sections, and the second full-time counselor taught four sec- 
tions. Thus, each case manager was involved in three or four learning communities and was re- 
sponsible for 75 to 100 students — though, as discussed above, the number of students per learn- 
ing community varied. 

The freshman orientation class is a standard course that is offered to all Kingsborough 
freshmen. All sections, whether in or out of Opening Doors Learning Communities, cover the 
same general topics, but staff reported that they exercise discretion over the specific reading and 
writing assignments and over how they manage the class. For example, one Opening Doors 
Learning Communities instructor/case manager reported that she gave four writing assignments, 
while others reported that they assigned less writing. Most of the case managers reported that the 
freshman orientation class was not very closely linked to the English and content courses at the 
curricular level. 

Teaching the freshman orientation class provided the case managers with an opportunity 
to see all their students at least once a week. However, all the case managers reported that they 
frequently saw students one-on-one in the program office, which had space for private discussions 
with students. One of the case managers required each student to schedule a formal in-person 
meeting with her within the first three weeks of the semester. The others reported that they at- 
tempted to meet with all their students at least once but did not enforce this policy rigidly. In addi- 
tion to scheduled meetings, all the case managers reported that they would reach out to students 
who were not attending class or when they learned from another faculty team member that a stu- 
dent had poor attendance or was having difficulty in the English or content course. All the case 
managers also encouraged students to visit the office whenever they had questions or needed as- 
sistance. 



The case managers described their role as a mix of academic advising and basic personal 
counseling. Academic advising included discussions about issues like what major to pursue or 
what courses to select, while counseling involved personal issues that may affect academic per- 
formance or progress. 

The academic advising role was particularly important for liberal arts majors. All Kings- 
borough students in their first semester must see an adviser as they select courses for the upcom- 
ing semester. Students with a major other than liberal arts would typically be assigned to a faculty 
member in the appropriate department, or might visit the Academic Advisement Center, which 
was established midway through the study period. Liberal arts majors, who are not connected to 
an academic department, would normally be advised by staff from the Freshman Year Experience 
office. However, Opening Doors Learning Communities students who were liberal arts majors 
were advised by their case manager/instructor. Case managers reported that they also provided 
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academic advising to many non-liberal arts majors because they had closer relationships with 
these students than the faculty advisers or staff from the Academic Advisement Center. 

Opening Doors Learning Communities case managers consistently made two general 
points in describing their counseling function. First, they noted that Kingsborough offers many 
specialized services for students, including a counseling center staffed by licensed psychologists, 
drug and alcohol counseling, a child care center, and a health services office. The case managers 
stressed that they were not trained to deal with these matters and saw their role as making referrals 
when serious issues emerged with Opening Doors students. All the case managers reported mak- 
ing numerous referrals to the counseling center. 

Second, the case managers reported that their counseling role was typically more exten- 
sive with students in the lower-level developmental English courses, who were seen as “less ma- 
ture” than students who tested into the higher-level English courses. 

In describing the types of issues that they addressed with students, case managers men- 
tioned stress and time management, helping students clarify their goals and reasons for attending 
college, and addressing inappropriate classroom behavior. The case managers also frequently 
helped students navigate college systems. For example, one case manager reported that a student 
who had a complicated question about financial aid could get an answer simply by walking into 
the financial aid office but that she could get it more quickly and more efficiently by making a call 
to someone she knew in that office. 

In interviews, most students spoke very positively about their Opening Doors case man- 
agers. For example, one student said: “She was wonderful, I have to say. She’s amazing. I was in 
her office every day. She was always there, and she always helped me.” 

Transitioning Students Out of the Program 

As noted above, the core features of Kingsborough’ s Opening Doors program operated 
only during the first 12-week module of a student’s first semester; the subsequent 6-week module 
served as a transition period. Students who attended classes during the shorter session continued to 
be assigned to their Opening Doors case manager — who would likely help them register for 
courses for the upcoming semester — and they received a second textbook voucher if they en- 
rolled in courses during the 6-week module. Kingsborough decided that there was no practical 
way to maintain the linked-course structure after the first module, since students needed and 
wanted to take a variety of different courses in subsequent semesters. 

Program administrators struggled to develop an appropriate way to ease students out of 
the highly structured, supportive environment of Opening Doors into the larger college commu- 
nity. During fall 2003, program staff organized a festive “graduation” ceremony to take place dur- 
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ing the 6-week winter module. Students received awards for completion of the program and were 
encouraged to meet the staff from the general freshmen counseling program. The event was suc- 
cessful, but attendance was uneven because only about half of the Opening Doors students had 
registered for classes during the module. In subsequent semesters, the event was held near the end 
of the 12-week module, while students were still on campus. 



How Did Opening Doors Learning Communities Differ from the 
Control Group Environment? 

Table 3.2 summarizes the key differences between Opening Doors Learning Communi- 
ties at Kingsborough and the regular college environment (that is, the control group environment). 
As expected, the contrast is clearest with respect to course assignments and scheduling: Opening 
Doors Learning Communities students took three linked courses that were scheduled in a block, 
and all of them took an English course and the freshman orientation class. As discussed above, the 
extent of integration across the linked courses varied from learning community to learning com- 
munity. Control group students took whatever courses were available to them (including, poten- 
tially, the same courses that were offered in Opening Doors), at whatever times those courses met, 
and were not required to take English or the freshman orientation. (Chapter 4 includes additional 
information about the courses that sample members took.) There was almost certainly no attempt 
by the regular college faculty to link the subject matter across courses. 

Class sizes also differed. Campuswide, a standard lecture class has about 35 students, on 
average, but developmental English courses average about 25 students per section. Although 
Opening Doors Learning Communities was intended to have a uniform class size of about 25 stu- 
dents, some of the sections were unusually small (Table 3.1). 

The textbook vouchers represent another clear contrast: Program group students had ac- 
cess to the vouchers (and generally used them), while control group students did not. Interestingly, 
though, in a small-scale survey administered to program group and control group students in fall 
2004, nearly equal percentages of students in the two groups reported that they had copies of nec- 
essary textbooks for their classes. 11 



"The Kingsborough Student Survey was adapted from the Pathways to Success Survey, created by Vin- 
cent Tinto. The Pathways to Success Survey is a modification of the Community College Survey of Student 
Engagement and was used with permission. 
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The Opening Doors Demonstration 
Table 3.2 



Key Differences Between the Opening Doors Learning Communities 
and the Regular College Environment During the First Semester 

Kingsborough Community College Report 



Feature 


Opening Doors Learning Communities 


Regular College Environment 


Block scheduling 


Cohort of up to 25 students took 3 courses 
together (in a group called a "learning 
community"); courses met one after the 
other. 


Students took courses whenever 
they were offered and available, 
with different students in each 
class. 


Curricular integration 


Curricula for the linked courses were 
integrated. 


There was no integration across 
courses. 


Class size 


Each course had a maximum of 25 
students. 


English courses typically had 25 
students; content courses 
averaged 30 to 35 students. 


Student development course 


All Opening Doors Learning Communities 
students took 1-credit freshman orientation 
class. 


Freshman orientation class was 
encouraged but not required. 


English courses 


All Opening Doors Learning Communities 
students took English, with the level being 
based on their scores on the City 
University of New York skills assessment 
tests. 


Students were encouraged, but 
not required, to take an English 
course, with the level being 
based on their scores on the City 
University of New York skills 
assessment tests. 


Tutoring 


Tutors were assigned to each learning 
community and attended classes. 


Tutors were assigned to 
developmental courses; 
otherwise, tutoring was accessed 
through central lab. 


Counseling 


An Opening Doors Learning Communities 
counselor was assigned to each learning 
community; each counselor was 
responsible for 75 to 100 students; 
counselors worked proactively to identify 
and resolve students' barriers to good 
attendance and academic performance. 


Students could access counseling 
on their own initiative; caseload 
for freshmen counselors was 
roughly 500:1; counseling role 
was reactive. 


Textbook voucher 


Voucher had a value of $225 during the 
first semester. 


No voucher was offered. 



SOURCE: MDRC field research data. 
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Enhanced tutoring was, in principle, another important difference between Opening 
Doors and the regular college environment. However, it is interesting to note that this component 
of the program attracted the most consistently negative reviews from program group students, 
many of whom resented having to work with tutors. 

It is more complicated to define the treatment difference with respect to student services. 
As noted above, Kingsborough offers an extensive array of specialized services for its students. 
There are also a number of programs targeted to particular populations, such as College Discov- 
ery, a CUNY-wide program for academically and financially disadvantaged freshmen. Moreover, 
Kingsborough’ s Freshman Year Experience program, established in 1998, is designed to help ori- 
ent students to the college and help them develop a coherent academic and career plan. Finally, 
midway through the study period, Kingsborough opened a new Academic Advisement Center — 
another collaborative project between academic affairs and student services that is designed to 
enhance the quality and quantity of advising. 

Given this extensive network of supports, it seems clear that students in both research 
groups had access to many of the same services. The key difference was that, in theory, the ser- 
vices in Opening Doors Learning Communities were more proactive and intensive. For example, 
although control group students were served by the Freshman Year Experience program, the ratio 
of students to staff was much higher than in Opening Doors Learning Communities. Moreover, 
staff for the Freshman Year Experience were not responsible for specific caseloads of students 
(thus, students would not necessarily see the same staff person at each office visit), and staff were 
not expected to reach out proactively to students who were having difficulty. Indeed, on the small- 
scale survey mentioned above, which was administered while Opening Doors Learning Commu- 
nities students were still in the program, some of the largest differences between groups are on 
questions that reflect the level of support that students received. For example, 62 percent of the 
program group members reported that there was someone at Kingsborough whom they could turn 
to for advice or support, compared with 39 percent of the control group. The Opening Doors 12- 
Month Survey, discussed in Chapter 4, found similar results. 
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Chapter 4 

Effects on Educational Outcomes 



As discussed in Chapter 1 , a key goal of learning communities is to increase educa- 
tional attainment among students, particularly those who are not well prepared for college- 
level work. Thus, the effect of learning communities on educational outcomes is central. 
This chapter focuses on educational outcomes as far as four semesters after an individual 
first enrolled in the Opening Doors study at Kingsborough Community College in Brook- 
lyn, New York. 1 These outcomes were measured using a survey conducted approximately 
12 months after individuals were randomly assigned either to the program group (which 
was eligible to enroll in a learning community) or to the control group (which received the 
college’s standard courses and services); transcript data from Kingsborough; skills assess- 
ment test data from all City University of New York (CUNY) campuses; and data on stu- 
dent enrollment from a national clearinghouse, which allows the analysis to account for the 
fact that many students enrolled at schools other than Kingsborough. 2 

The chapter first discusses the impact of being assigned to the program group on the 
students’ educational experiences and outcomes, as measured by data from Kingsborough 
during their first semester in the study (the “program semester”); then it presents the im- 
pacts on their educational outcomes at Kingsborough during the following three semesters. 
Next, the chapter discusses the effects on a student’s progression through the English re- 
quirements at Kingsborough, including the impacts on CUNY reading and writing assess- 
ment tests. The discussion then considers more general educational outcomes (including 
enrollment at institutions beyond Kingsborough) in at least one of the first four semesters 
after random assignment. The chapter’s final section briefly discusses program effects for 
some subgroups of sample members. 

The key findings are: 

• Assignment to the program group significantly improved a student’ s college 
experience in terms of a sense of integration, belonging, and the value of the 
skills acquired, suggesting that learning communities change a student’ s per- 
ceptions of the college experience. 



'in results not presented here, analysis of the Opening Doors 12-Month Survey suggests that there were no 
meaningful impacts of assignment to the program group on employment outcomes. This result is not surprising, 
given that the students in the study were so young that one would not expect to observe such impacts in so short a 
period of time. 

2 The data sources are described in Chapter 2. 
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• Further, the results suggest that learning communities improve educational 
outcomes along a number of dimensions in the program semester. For exam- 
ple, assignment to the program group increased the number of courses stu- 
dents attempted, whether they passed all courses, and the number of credits 
earned in the program semester. 

• The evidence also indicates that students in learning communities are more 
likely to attempt both reading and writing CUNY assessment tests and were 
more likely to have passed them two semesters after completion of the learn- 
ing community. 3 

• More generally, the results suggest that learning communities provide a 
short-term boost to educational attainment that subsequently plateaus, as the 
educational gains made by Kingsborough students who were assigned to the 
program group — including gains on CUNY assessment tests — did not 
generally increase after the program semester. 4 As an exception, however, 
there is evidence that learning communities may increase student persistence 
— in terms of college enrollment — over time. 

• Impacts were examined for some different subgroups of students, and, for the 
most part, no meaningful differences across groups were found. The magni- 
tude of several of the impacts is larger for men than for women, but most of 
the differences between the two groups are not statistically significant. 

As described more fully in Box 4.1, the tables that follow present outcomes for the 
students assigned to the program group (the treatment group), outcomes for the control 
group, the difference between the two groups (the impact of the program), the standard er- 
ror of the difference, and the estimated effect size. These means are adjusted for the stu- 
dent’ s cohort — which reflects the point at which the student was randomly assigned to the 
program group or control group — as well as the number of English assessment tests passed 
at baseline (random assignment). No other covariates are included. 5 



3 Similar results on impacts using the first cohort are reported in Bloom and Sommo (2005). One must keep in 
mind that this analysis does not isolate the impact of learning communities on passing an assessment test among 
students who attempt one. Rather, it reflects a combined impact of learning communities on attempting a test and 
their effect on passing it among those students who attempt it. 

4 One must exercise caution when interpreting these results, since part of any educational improvement ob- 
served at Kingsborough in semesters after the program semester is due, in part, to the fact that students in the 
learning communities were more likely to reenroll at Kingsborough in the first place. 

5 Nearly all the results are qualitatively similar if other covariates are included. Results without (other) covari- 
ates are presented because, with a randomized design, it is unnecessary to include them; the coefficient estimates 
are unbiased. Further, it is not at all clear how one decides which covariates to include. The number of baseline 

(continued) 
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The Impact on Student Perceptions of College Experiences: 
Results from the Opening Doors Twelve-Month Survey 

In order to properly interpret the impact of learning communities on student out- 
comes, one must first attempt to determine whether the study changed the educational ex- 
periences of those students randomly assigned to the program group. Since, ideally, a fully 
developed learning community involves an integrated curriculum between the linked 
courses that the students take together (see Chapter 3, Figure 3.1), one would expect that 
students who were assigned to the program group would feel a better sense of belonging 
and integration into the school and would be more engaged with their studies. To gauge 
students’ perceptions, the 12-month survey asked respondents a battery of questions about 
their educational experiences at Kingsborough. 6 The components of each battery were then 
summed to create four scales: integration and sense of belonging at school, participation 
and engagement at school, using knowledge (critical thinking curriculum), and acquired 
academic and work skills. 7 Box 4.2 briefly describes each scale and provides a subset of its 
items; Appendix C describes the full battery of items. 

Table 4.1 presents the impacts of learning communities on these four measures as 
well as whether the respondent rated the college experience as “good or excellent,” how 
much time the respondent spent on campus in the first (program) semester, and how much 
time the respondent studied during the first semester. To facilitate interpretation of the scale 
scores, the table presents the fraction of students whose scaled score is either “high” (mean- 
ing it is 1 standard deviation above the average) or “low” (meaning it is 1 standard devia- 
tion below the average). 

The table shows that, among the survey respondents, those who were assigned to 
the program group had a better overall college experience, had more connections with other 



English assessment tests passed is included because it explains a significant portion of the variation in test score 
outcomes in Tables 4.5 and 4.6 and, therefore, substantially decreases the estimated standard errors (without 
changing the estimated impacts); their inclusion does not change other estimated impacts (or their standard errors). 
It should also be noted that while students who were assigned to a learning community were less likely to be de- 
pressed (according to the K6 Screening Scale developed by Dr. Ronald Kessler at Harvard Medical School) and 
were less likely to smoke at baseline (and these differences are statistically significant), the results in this chapter 
are similar if the impacts are adjusted (controlled for in an Ordinary Least Squares [OLS] regression) for these 
baseline differences. Finally, the standard errors in all the tables account for the clustering of students into learning 
communities. 

6 See Appendix B for an analysis of the response rates for the Opening Doors 12-Month Survey. 

7 Factor analysis was performed on each potential scale to determine how the items lined up along factors or 
dimensions. Eigenvalues and screen plots were used to help determine whether there were multiple subscales 
within given scales, and the Cronbach statistics for the fall scale were compared with those from the potential 
subscales. Final scales were constructed using these results. The scale items come from the Community College 
Survey of Student Engagement and were used with permission. 
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Box 4.1 



How to Read the Tables in This Report 

Most tables in this report use a similar format, illustrated below. The abbreviated table shows some 
educational outcomes for the program group and the control group. The first row, for example, 
shows that 93.1 percent of the program group members and 91.4 percent of the control group 
members registered for any courses at Kingsborough during their first semester in college (called 
the “program semester”). 

Because individuals were assigned randomly either to the program group or to the control group, 
the effects of the Opening Doors Learning Communities program can be estimated by the differ- 
ence in outcomes between the two groups. The “Difference” column in the table shows the differ- 
ences between the two research groups’ outcomes — that is, the program’s estimated impacts on 
the outcomes. For example, the estimated impact on registering for any courses at Kingsborough 
can be calculated by subtracting 91.4 percent from 93.1 percent, yielding an increase of 1.7 per- 
centage points. 

Differences marked with asterisks are “statistically significant,” meaning that it is quite unlikely 
that the program had no impact. The number of asterisks indicates whether the impact is statisti- 
cally significant at the 1 percent, 5 percent, or 10 percent level. (The lower the level, the less likely 
the estimated impact arises from a program with no true effect. One asterisk corresponds to the 10 
percent level; two asterisks, the 5 percent level; and three asterisks, the 1 percent level.) For exam- 
ple, as the second row below shows, the Opening Doors program increased the l ik elihood that a 
student enrolled in a learning community at Kingsborough by 84.6 percentage points, a difference 
that is statistically significant at the 1 percent level. 

The two rightmost columns of the impact tables in this report provide more information about the 
differences. The standard error of the impact estimate is the measure of uncertainty associated with 
it, and this is used to calculate the statistical significance of the impact. The effect size provides a 
way to interpret the substantive significance of an effect. It is calculated as the difference between 
the outcomes for the program group and control group, divided by the standard deviation of the 
control group. Thus the effect size “standardizes” the impacts across measures. 



The Opening Doors Demonstration 
Program Semester Transcript Outcomes 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Registered for any courses (%) 


93.1 


91.4 


1.7 


1.6 


0.06 


Enrolled in a learning community 3 


85.3 


0.7 


84.6 *** 


2.3 


10.50 


Number of courses attempted 


4.9 


4.4 


Q 4 *** 


0.1 


0.22 


Number of credits attempted 


15.7 


15.2 


0.5 


0.3 


0.08 


Regular credits 


10.0 


10.2 


-0.3 


0.3 


-0.05 


Equated credits 


5.7 


5.0 


0.8 ** 


0.3 


0.16 








Box 4.2 



Behind the Scales Measuring Classroom and College Experiences 

Integration and Sense of Belonging 

To gauge their sense of integration or belonging at the school, respondents were asked to rate 
whether they agreed or disagreed (on a scale of 1 to 4) with the following types of statements: I 
do not feel that I fit in or belong at this campus; the instructors and staff understand who I am, 
where I am coming from; I do not feel I am part of campus life; I know my way around this 
place. 

Participation and Engagement at School 

To measure each respondent’s participation and engagement at the school, respondents were 
asked how often they had done the following types of activities (also on a scale from 1 to 4): 
Made a class presentation, worked on a paper or project that required integrating ideas or in- 
formation from different classes, worked with classmates outside of class to prepare assign- 
ments, used e-mail to communicate with an instructor, discussed ideas from your readings or 
classes with instructors outside of class, worked with instructors on activities other than 
coursework. 

Using Knowledge (Critical Thinking Curriculum) 

To gauge the extent to which the curriculum in the learning communities developed critical 
thinking, respondents were asked to rate how much their coursework emphasized the following 
activities (on a scale from 1 to 4): analyzing the basic elements of an idea, experience, or the- 
ory; synthesizing and organizing ideas, information, or experiences in new ways; making 
judgments about the value or soundness of information, arguments, or methods; integrating 
ideas, information, or skills from different classes. 

Acquired Academic and Work Skills 

Respondents’ ratings of how much their college experiences had affected them in the following 
areas were used to measure the extent to which they had acquired valuable academic and work 
skills in class (on a scale from 1 to 4): acquiring a broad general education, acquiring job or 
work-related knowledge and skills, writing clearly and effectively, thinking critically and ana- 
lytically, learning effectively on your own, developing clearer career goals, developing a sense 
of confidence in your academic abilities. 
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students and faculty, were more engaged in the classroom, and experienced a curriculum 
that required more critical thinking. For example, 17.3 percent of students in the control 
group rated their sense of integration on campus as “low,” compared with only 11.4 percent 
of students in the program group. The difference of 5.9 percentage points between the learn- 
ing communities group and the control group reflects an effect size of 16 percent (or a de- 
crease of 16 percent of a standard deviation) that they generally disagreed with the state- 
ments of feeling integrated. Importantly, this decrease is also statistically significant, mean- 
ing that it did not likely occur by chance. 

More generally, learning communities appear to have had their greatest impact on a 
student's reporting having a poor experience (as proxied by having a “low” scale score on 
one of the measures). Students in the program group were 7.3 percentage points less likely 
to report “low” engagement, 5.9 percentage points less likely to report a “low” use of 
knowledge in their courses, and 5.6 percentage points less likely to report “low” levels of 
acquiring academic and work skills. When asked about their overall college experience, 
83.2 percent of students in the program group rated their experience “good or excellent,” 
compared with 76.2 percent of the control group students. Importantly, all these impacts are 
statistically significant. 

Table 4.1 also reports that students in the program group spent slightly less time on 
campus than students in the control group. For example, the program group students spent, 
on average, 10.7 hours per week on campus in the first semester, compared with an average 
of 10.9 hours per week on campus for control group students. Similarly, program group 
students were 4.5 percentage points more likely to report spending 12 or fewer hours per 
week on campus in the first semester. While these differences are not statistically signifi- 
cant, they may seem surprising at first glance. However, as discussed in Chapter 3, when 
scheduling the courses that make up the learning communities, administrators aimed to 
group the classes together in order to minimize the time that students had to spend on cam- 
pus between classes. The results suggest only a small difference between the program and 
control groups in the time that students spent on campus, suggesting either that the students 
in the control group were also able to schedule their classes effectively or that although stu- 
dents in the program group may have spent less time on campus as a result of their class 
schedule, they may have spent more time on campus for other reasons. Finally, Table 4.1 
reports that students in the program group reported spending the same number of hours 
studying per week in the first semester as those in the control group. 

The findings suggest that students who were randomly assigned to the program 
group were better integrated into the college, were more likely to participate in class, felt 
more engaged at Kingsborough, and were more likely to feel that the college curriculum 



54 




The Opening Doors Demonstration 

Table 4.1 

Classroom and College Experiences of Sample Members 



Kingsborough Community College Report 



Outcome 


Sample 

Size 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Integration and sense of 
belonging at school 2 (%) 
Low 


984 


11.4 


17.3 


_5 9 *** 


2.1 


-0.16 


High 


984 


15.6 


12.6 


3.0 


2.1 


0.09 


Participation and engagement 13 (%) 
Low 


1,008 


14.8 


22.1 


_7 3 *** 


2.5 


-0.18 


High 


1,008 


18.0 


12.4 


5.5 ** 


2.4 


0.17 


Using knowledge (critical 
thinking curriculum ) 2 (%) 
Low 


1,012 


12.3 


18.1 


_5 9 *** 


2.2 


-0.15 


High 


1,012 


24.4 


22.4 


2.0 


2.5 


0.05 


Acquired academic and work skills ' 1 (%) 
Low 


1,005 


12.5 


18.1 


-5.6 ** 


2.2 


-0.15 


High 


1,005 


21.1 


16.3 


4.8 ** 


2.3 


0.13 


Rated college experience 
good or excellent (%) 


1,015 


83.2 


76.2 


7 i *** 


2.5 


0.17 


Hours per week spent on 
campus in first semester 


1,074 


10.7 


10.9 


- 0.2 


0.3 


-0.03 


Spent 1 2 or fewer hours per week 
on campus in first semester (%) 


1,074 


42.4 


37.9 


4.5 


3.1 


0.09 


Hours per week studying 
in first semester 


1,068 


7.6 


7.6 


0.0 


0.4 


- 0.01 


Studied 19 or more hours per week 
in first semester (%) 


1,068 


8.7 


9.1 


-0.5 


1.8 


- 0.02 



(continued) 
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Table 4.1 (continued) 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

a 10-item scale about sense of integration with and belonging to the school community; response categories 
range from 1 = "strongly disagree" to 4 = "strongly agree." "Low” is the percentage of sample members scoring 
one standard deviation below the mean; "high” is the percentage of sample members scoring one standard 
deviation above the mean. 

b 15-item scale about participation in schoolwork, projects, and ideas; response categories range from 1 = 
"very often" to 4 = "never." "Low" is the percentage of sample members scoring one standard deviation below 
the mean; "high" is the percentage of sample members scoring one standard deviation above the mean. 

c 6-item scale about using aquired knowledge inside and outside the classroom; response categories range 
from 1 = "very much" to 4 = "very little." "Low" is the percentage of sample members scoring one standard 
deviation below the mean; "high" is the percentage of sample members scoring one standard deviation above 
the mean. 

d 16-item scale about aquiring academic and work skills, and a sense of self and community; response 
categories range from 1 = "very much" to 4 = "very little." "Low" is the percentage of sample members scoring 
one standard deviation below the mean; "high” is the percentage of sample members scoring one standard 
deviation above the mean. 



furthered their critical thinking and that they had acquired valuable academic and work- 
related skills. The fact that these are precisely the goals of learning communities suggests 
that while implementation at Kingsborough may not have been perfect (see Chapter 3), it 
was still strong; assignment to the program group appears to have significantly changed the 
educational experiences of students, compared with the “regular” college experience at 
Kingsborough. 

While the results in this section suggest that learning communities improve the edu- 
cational experience of students, they do not establish that these experiences translate into 
better academic achievement as well. The next two sections present the impact of assign- 
ment to the program group (and, therefore, the impact of learning communities) on meas- 
ures of academic achievement during the program semester and beyond. 
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Educational Outcomes During the Program Semester 

The next question is whether assignment to the program group improved the aca- 
demic outcomes of students during the semester when they were in a learning community 
(called the “program semester”). Table 4.2 presents these results using transcript data from 
Kingsborough. 8 In this first semester after random assignment, 93.1 percent of the program 
group and 91.4 percent of the control group registered for any courses at Kingsborough. 
(Thus, from 7 percent to 9 percent of each group elected not to enroll in the college in the 
first semester after random assignment, which, as discussed in Chapter 2, occurred just be- 
fore registration.) While students in the program group were slightly more likely to enroll in 
this first semester, the difference is not statistically significant. In addition, over 85 percent 
of the students in the program group enrolled in a learning community during the program 
semester, compared with less than 1 percent of the control group. 9 

Notably, a positive impact of learning communities on academic achievement is 
evident in several measures shown in Table 4.2. For example, the students in the pro- 
gram group attempted just less than one-half of a course more (for an average course 
load of about 4.9 courses). In addition, only about 33 percent of the control group and 
43 percent of the program group actually passed all of their courses. Students in the 
program group were, therefore, 10 percentage points more likely to pass all of their 
courses and were nearly 8 percentage points less likely to withdraw from any courses; 
they earned one more credit, on average, during this first semester. The distribution of 
grade point averages (GPAs) also generally suggests that the program group students’ 
were higher averages. Strikingly, however, there is no statistical difference in the num- 
ber of total credits attempted, largely be-cause the program group attempted slightly 
fewer regular credits and slightly more developmental (or “equated”) credits. 

Overall, the results in Table 4.2 suggest that, on average, students’ educational 
outcomes significantly improve during a semester in which they are participating in a 
learning community. 10 



s Richburg-Hayes, Visher, and Bloom (2008) present some findings from the Opening Doors study at Kings- 
borough. Some of the findings differ somewhat from those presented in this report, because the analyses for the 
article was based on fewer semesters of data. 

9 The analysis in Table 4.2 indicates that 93.1 percent of students in the program group registered for classes 
at Kingsborough, while only 85.3 percent actually enrolled in a learning community. The difference occurs for 
several reasons, such as that the students changed their minds, that their class schedule could not accommodate the 
learning community, or that they were assigned to a particular learning community but it was subsequently deter- 
mined that their placement was not correct. 

10 One must keep in mind that most of the results in Table 4.2 require careful interpretation, as the outcomes 
are observed only for students who register or enroll at Kingsborough. In order to keep the analysis straightfor- 

(continued) 
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Educational Outcomes After the Program Semester 

In assessing whether the impact of learning communities persists after students 
leave that unique educational experience, two types of outcomes were considered. The first 
type of outcome reflects the educational attainment of students based on records from 
Kingsborough (and, for the analysis on student success on assessment tests, from CUNY 
records as well). The value of these data is that they are quite rich, with many different 
types of educational outcomes. The disadvantage is that to the extent students may have left 
Kingsborough (or the CUNY system) and enrolled at other educational institutions, the im- 
pacts that occurred at Kingsborough (or CUNY) may not fully reflect the impact of learning 
communities on educational attainment more broadly defined. As such, data from a wide 
range of institutions were also analyzed to assess whether learning communities improve 
educational attainment beyond Kingsborough. 

Overall, one does not observe a large impact of learning communities on college en- 
rollment and numbers of credits attempted and earned in the first two semesters after the 
program semester (which are called “postprogram semesters”). In contrast, there appears to 
be an impact on enrollment in the third postprogram semester. Through the second postpro- 
gram semester, students in learning communities are more likely to attempt both reading 
and writing assessment tests and are more likely to have passed them. One must keep in 
mind that this analysis does not isolate the impact of learning communities on passing an 
assessment test among students who attempt one. Rather, it reflects a combined impact of 
learning communities on attempting a test and of passing it among those who attempt one. 



ward, outcomes have been effectively “imputed” for those students who did not register or enroll in any particular 
semester. For example, among the students who “did not pass all courses at Kingsborough” or who “withdrew 
from any courses” are those who did not attempt any courses, either. Strictly speaking, these students did not take 
any courses at Kingsborough, and, therefore, they did not pass all of their courses or withdraw from any courses. 
As a result, however, part of the estimated differences in outcomes reflects the fact that students in the control 
group were less likely to register for any courses and were less likely to attempt any courses in the first place. 
Similarly, those who did not attempt any courses at Kingsborough have “No GPA at Kingsborough,” so that part 
of the improvement in GPA that is associated with being assigned to a learning community is due to an increased 
likelihood of attending Kingsborough, rather than simply to doing better in courses in which students are enrolled. 
Unfortunately, this latter impact — the effect of being assigned to the program group on course success among 
those who chose to enroll at Kingsborough — is not straightforward to estimate. 
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The Opening Doors Demonstration 
Table 4.2 

Transcript Outcomes: Program Semester 



Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Registered for any courses (%) 


93.1 


91.4 


1.7 


1.6 


0.06 


Enrolled in a learning community 3 (%) 


85.3 


0.7 


84.6 *** 


2.3 


10.50 


Number of courses attempted 


4.9 


4.4 


Q 4 *** 


0.1 


0.22 


Number of credits attempted 


15.7 


15.2 


0.5 


0.3 


0.08 


Regular credits 


10.0 


10.2 


-0.3 


0.3 


-0.05 


Equated credits 


5.7 


5.0 


0.8 ** 


0.3 


0.16 


Passed all courses (%) 


43.1 


33.0 


| *** 


2.5 


0.22 


Number of courses passed 


3.8 


3.2 


0.6 *** 


0.1 


0.30 


Withdrew from any courses (%) 


26.6 


34.5 


_7 g *** 


2.3 


-0.17 


Number of course withdrawals 


0.5 


0.7 


-0.2 ** 


0.1 


-0.13 


Number of credits earned 


11.5 


10.4 


1 2 *** 


0.4 


0.16 


Regular credits 


8.0 


7.7 


0.3 


0.3 


0.05 


Equated credits 


3.5 


2.6 


q g *** 


0.3 


0.22 


Term GPA (%) 












No GPA b 


8.7 


11.8 


-3.1 * 


1.8 


-0.09 


3.0 to 4.0 or B/A 


34.5 


27.7 


6.8 ** 


2.7 


0.15 


2.0 to 2.9 or C/B- 


30.3 


28.0 


2.3 


2.2 


0.05 


1.0 to 1.9 or D/C- 


15.0 


17.5 


-2.5 


1.8 


-0.07 


0 to 0.9 or F/D- 


11.6 


15.0 


-3.5 ** 


1.7 


-0.10 


Sample size (total = 1,534) 


769 


765 









SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

3 While most students were enrolled in all three linked courses, a small number withdrew from part of 
the link. 

b The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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Kingsborough- and CUNY-Specific Educational Outcomes 

Table 4.3 presents the academic outcomes from Kingsborough transcript data for 
the first, second, and third postprogram semesters (that is, for the second, third, and fourth 
semesters after random assignment). A common pattern among community colleges that is 
also readily apparent in these data is that, after the program semester, there is substantial 
attrition from Kingsborough for students in both the program and the control group. For 
example, in the first postprogram semester, only about three-quarters of the students in the 
study registered for any classes at Kingsborough; in the second postprogram semester, only 
60 percent registered; and in the third postprogram semester, only about one-half registered 
for any courses. In each of these semesters, while the students in the program group were 
more likely to register than those in the control group, the difference between the two 
groups is statistically significant only in the third postprogram semester, when students in 
the program group were 5 percentage points more likely to register — an 1 1 percent in- 
crease over the control group. 

In addition, during the first postprogram semester, students in the program group at- 
tempted and earned slightly more regular credits than students in the control group. Specifically, 
the program group attempted and earned about one -half more regular credits than the control 
group, although only the impact on attempted credits is statistically significant. There is virtu- 
ally no difference in equated credits between the two groups: Members of the control group at- 
tempted 2.3 equated credits, compared with 2.2 attempted equated credits among members of 
the control group. Each group earned about 1 equated credit. The fact that assignment to the 
program group had a greater impact on regular credits than equated credits in the first postpro- 
gram semester suggests that the intervention may initially help students to move beyond devel- 
opmental courses — an outcome studied directly in Table 4.6, below. 

In the second and third postprogram semesters, these gains faded. For example, in 
the second postprogram semester, while students in the program group attempted just under 
one-half more regular credits than those in the control group, this difference is not statisti- 
cally significant. Further, there is virtually no difference in the number of regular credits 
actually earned between the two groups of students. Importantly, students who had been 
assigned to the program group attempted 9.9 total credits and earned 7.1 total credits, com- 
pared with totals of 9.6 attempted credits and 6.9 earned credits among students in the con- 
trol group. These differences are so small that they are not educationally meaningful, and 
they may have occurred by chance. 11 



"in addition, part of any educational improvement observed at Kingsborough in postprogram semesters is 
due to the fact that students in the program group were more likely to reemoll at Kingsborough in the first place. 

(continued ) 
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Cumulative impacts reflecting the program semester through the third postprogram 
semester are reported in Table 4.4. Over the course of four semesters at Kingsborough, the 
program group registered for slightly more semesters and earned 2.4 more credits than the 
control group. However, these educational gains are primarily due to the impacts observed 



Unfortunately, as noted in the preceding footnote, it is not straightforward to estimate the impact of learning 
communities on these subsequent outcomes that is net of their effect on enr ollment at Kingsborough. 
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The Opening Doors Demonstration 
Table 4.3 

Transcript Outcomes: First, Second, and Third Postprogram Semesters 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


First postproaram semester 


Registered for any courses (%) 


77.4 


75.0 


2.4 


2.1 


0.05 


Number of credits attempted 


12.5 


11.8 


0.7 * 


0.4 


0.09 


Regular credits 


10.2 


9.6 


0.6 * 


0.3 


0.08 


Equated credits 


2.3 


2.2 


0.1 


0.3 


0.02 


Number of credits earned 


8.6 


8.0 


0.6 * 


0.3 


0.09 


Regular credits 


7.5 


7.0 


0.5 


0.3 


0.08 


Equated credits 


1.1 


1.0 


0.1 


0.2 


0.03 


Term GPA (%) 


No GPA a 


24.8 


26.3 


-1.5 


2.1 


-0.03 


3.0 to 4.0 or B/A 


22.7 


23.0 


-0.3 


2.1 


-0.01 


2.0 to 2.9 or C/B- 


25.7 


22.2 


3.5 * 


2.0 


0.08 


1.0 to 1.9 orD/C- 


13.1 


15.4 


-2.3 


1.8 


-0.06 


0 to 0.9 or F/D- 


13.7 


13.1 


0.6 


1.6 


0.02 


Second postproaram semester 


Registered for any courses (%) 


61.3 


59.2 


2.0 


2.4 


0.04 


Number of credits attempted 


9.9 


9.6 


0.2 


0.4 


0.02 


Regular credits 


8.7 


8.3 


0.4 


0.4 


0.05 


Equated credits 


1.1 


1.3 


-0.2 


0.2 


-0.07 


Number of credits earned 


7.1 


6.9 


0.1 


0.4 


0.02 


Regular credits 


6.6 


6.3 


0.2 


0.4 


0.04 


Equated credits 


0.5 


0.6 


-0.1 


0.1 


-0.06 


Term GPA (%) 


No GPA a 


40.1 


41.7 


-1.6 


2.5 


-0.03 


3.0 to 4.0 or B/A 


17.6 


20.4 


-2.8 


2.0 


-0.07 


2.0 to 2.9 or C/B- 


22.2 


17.1 


5.1 ** 


2.0 


0.14 


1.0 to 1.9 or D/C- 


10.4 


13.0 


-2.6 


1.9 


-0.08 


0 to 0.9 or F/D- 


9.7 


7.9 


1.9 


1.2 


0.07 


Third postproaram semester 


Registered for any courses (%) 


52.9 


47.8 


5.1 * 


2.7 


0.10 


Number of credits attempted 


7.9 


7.2 


0.7 * 


0.4 


0.09 


Regular credits 


7.1 


6.5 


0.6 


0.4 


0.09 


Equated credits 


0.7 


0.7 


0.1 


0.1 


0.03 



(continued) 
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Table 4.3 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Number of credits earned 


6.0 


5.4 


0.5 


0.4 


0.08 


Regular credits 


5.6 


5.1 


0.5 


0.3 


0.07 


Equated credits 


0.3 


0.3 


0.0 


0.1 


0.03 


Term GPA (%) 


No GPA a 


48.7 


53.5 


-4.8 * 


2.7 


-0.10 


3.0 to 4.0 or B/A 


16.7 


17.5 


-0.8 


1.9 


-0.02 


2.0 to 2.9 or C/B- 


19.0 


15.6 


3.4 * 


2.0 


0.09 


1.0 to 1.9 or D/C- 


9.5 


8.5 


1.0 


1.5 


0.04 


0 to 0.9 or F/D- 


6.1 


5.0 


1.2 


1.2 


0.05 


Sample size (total = 1,534) 


769 


765 









SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

‘The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 



during the program semester (and, to some extent, during the first postprogram semester). 
In addition, there is no discernable educationally meaningful impact on cumulative GPA. 

Progression Through English Requirements 

As discussed in Chapter 1, a goal of this program for students who failed one or 
both of the English skills assessment tests at baseline is to help them move more quickly 
through developmental English requirements, allowing them to take college-level English. 
To do so, a student first must pass the reading and writing assessment tests, and evidence 
that learning communities increase passing rates would be an important sign of their suc- 
cess. Analysis of the impact of learning communities on student assessment test outcomes is 
presented in Table 4.5. Drawn from all CUNY campuses, the assessment data reflect 
broader measures of educational success than can be obtained from the Kingsborough tran- 
script data alone. 
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The Opening Doors Demonstration 
Table 4.4 



Cumulative Outcomes: Program Semester Through Third Postprogram Semester 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Registered for any courses (%) 


95.2 


94.2 


1.0 


1.3 


0.04 


Number of semesters registered 


2.8 


2.7 


0.1 * 


0.1 


0.09 


Number of credits earned 


33.2 


30.8 


2 4** 


1.2 


0.10 


Regular credits 


27.7 


26.2 


1.6 


1.1 


0.07 


Equated credits 


5.5 


4.6 


0.9 * 


0.5 


0.14 


Cumulative GPA a (%) 


No GPA 


7.3 


9.0 


-1.8 


1.5 


-0.06 


3.0 to 4.0 or B/A 


25.2 


24.3 


0.9 


2.2 


0.02 


2.0 to 2.9 or C/B- 


33.7 


29.6 


4.1 * 


2.2 


0.09 


1.0 to 1.9 orD/C- 


19.7 


20.4 


-0.7 


2.0 


-0.02 


0 to 0.9 or F/D- 


14.0 


16.6 


-2.6 


1.9 


-0.07 


Sample size (total = 1,534) 


769 


765 









SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

“'Cumulative GPA is based on credit-bearing courses taken from random assignment through the end of 
the third postprogram semester. Courses in which students did not receive a passing grade and that they 
subsequently repeated are not included in the cumulative GPA, as per Kingsborough Community College 
policy. 

b The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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The upper panel of Table 4.5 assesses the impact of assignment to the program group on 
assessment test outcomes during the program semester. The results suggest that 13.0 per- 
cent of the program group attempted a reading assessment test in the first semester, com- 
pared with 9.4 percent of the control group. Further, a slightly higher percentage of the pro- 
gram group had passed the reading test by the end of the first semester (83.8 percent of the 
program group, compared with 81.2 percent of the control group). Neither of these impacts, 
however, is statistically significant. 

While there is not a great impact on reading assessment tests — perhaps because so 
many of the students had passed the test at baseline — there appears to be a larger impact 
on attempting a writing test. During the program semester, students in the program group 
were nearly 1 1 percentage points more likely to attempt a writing assessment test — an im- 
pact that is statistically significant. Given that students must have successfully completed a 
developmental English course before being eligible to retake the assessment test, the find- 
ing of an impact on attempting the test is noteworthy. Students in the program group were 
also 6.3 percentage points more likely to have passed the writing test and 6.0 percentage 
points more likely to have passed both tests during this first semester — differences that are 
also statistically different from zero. 

The lower panel of Table 4.5 presents the impact of assignment to the program 
group on assessment test outcomes in the first three semesters after random assignment 
(that is, the program semester plus two postprogram semesters). 12 In total, students in the 
program group were more likely to have attempted either a reading or a writing test, made 
more attempts to pass the tests, and were more likely to have passed the tests; and these dif- 
ferences are statistically significant. Most of these impacts, however, appear to be driven by 
results from the program semester. In addition, one must keep in mind that interpreting the 
effects on passing assessment tests includes the impact of being assigned to the program 
group on test-taking attempts. That is, the greater percentage of program group students 
who passed the writing test, for example, may be due, in part, to more test-taking attempts. 



12 When this report was written, assessment data were not available for later semesters. 
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Table 4.5 



English Skills Assessment Test Outcomes 
Kingsborough Community College Report 



Outcome (%) 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Prosram semester 












Attempted reading test 


13.0 


9.4 


3.7 


2.5 


0.13 


Passed reading test by end of semester 


83.8 


81.2 


2.6 


1.7 


0.07 


Attempted writing test 


36.2 


25.7 


10.5 *** 


3.6 


0.24 


Passed writing test by end of semester 


53.3 


47.0 


6.3 ** 


2.8 


0.13 


Attempted either English skills assessment test 


41.5 


30.9 


10.6 *** 


3.9 


0.23 


Passed both English skills 
assessment tests by end of semester 


52.3 


46.3 


6.0 ** 


2.7 


0.12 


Program and nostnrogram semesters' 1 












Attempted reading test 


17.8 


13.3 


4 5 *** 


1.7 


0.13 


Passed reading test by end 
of second postprogram semester 


88.6 


86.0 


2.5 * 


1.5 


0.07 


Attempted writing test 


46.8 


39.1 


7 7 *** 


2.5 


0.16 


Passed writing test by end 
of second postprogram semester 


66.2 


60.5 


5.7 ** 


2.7 


0.12 


Attempted either English skills assessment test 


50.5 


42.0 


8.5 *** 


2.5 


0.17 


Passed both English skills assessment tests 
by end of second postprogram semester 


65.2 


60.0 


5.2 * 


2.7 


0.11 


Sample size (total = 1,534) 


769 


765 









SOURCE: MDRC calculations from City University of New York skills assessment test data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

“Outcomes include data from the program semester through the second postprogram semester. 
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The second aspect of a student’s progression through English requirements is en- 
rollment and completion of regular (credit-bearing) English courses — Freshman English I 
and Freshman English II — that are required for completion of an associate’s degree. As 
discussed in Chapter 1, Kingsborough students cannot take the regular English courses until 
they have passed both the reading and the writing skills assessment tests. Table 4.6 presents 
the percentage of students in the program and control groups that were enrolled in Devel- 
opmental English, Freshman English I, and Freshman English II in the program semester 
and in the first and second postprogram semesters. The analysis is conducted separately for 
students who had failed both English tests at baseline, those who had failed only one Eng- 
lish test at baseline, and those who had passed both English tests at baseline. If learning 
communities accelerate progression through these requirements, as intended, one would 
expect to see declining proportions of students in the program group — relative to the con- 
trol group — who enrolled in Developmental English in subsequent semesters and increas- 
ing relative proportions who enrolled in the regular English courses. In all cases, learning 
communities should increase the relative proportions of program students who have passed 
the English courses, as these are cumulative measures. The data on course enrollment and 
passage reflect those measures only at Kingsborough; the data on having passed the English 
skills assessment tests are based on CUNY-wide information. 

Consider first students who had failed both English assessment tests at baseline. 
Students in the program group were nearly 17 percentage points more likely to enroll in 
Developmental English during the program semester than students in the control group. 
Further, they were 5 percentage points more likely to have passed both English assessment 
tests as of the end of the program semester — a result that mirrors the overall impacts on 
assessment tests in Table 4.5, although it is not statistically significant. In the first postpro- 
gram semester, the students in the program group were 12.5 percentage points more likely 
to be enrolled in Developmental English and 9.2 percentage points more likely to have 
passed both English assessment tests. By the second postprogram semester, they were less 
likely to be enrolled in Developmental English but were more likely to be enrolled in 
Freshman English I. However, because these differences are not statistically significant, 
they are only suggestive. 

There is a roughly similar pattern among those who had passed one of the English 
assessment tests at baseline. Students in the program group were 14.2 percentage points 
more likely to be enrolled in Developmental English during the program semester; they 
were also 5.2 percentage points more likely to be enrolled in Freshman English I. Similarly, 
these students were 10.2 percentage points more likely to have passed Developmental Eng- 
lish and 5.8 percentage points more likely to have passed Freshman English I, compared 
with the control group students. These impacts are all statistically significant. These impacts 
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Table 4.6 

English Progression, by English Skills Assessment at Baseline 



Kingsborough Community College Report 





Program 


Control 




Standard 


Effect 


Outcome (%) 


Group 


Group Difference 


Error 


Size 


Failed both English tests at baseline 












Program semester 












Enrolled in one or more English courses 3 












Developmental English 


87.3 


70.8 


16.6 *** 


4.8 


0.36 


Freshman English I 


1.0 


0.5 


0.5 


0.8 


0.07 


Freshman English II 


0.0 


0.0 


0.0 


NA 


NA 


Passed by end of semester 












Both English skills assessment tests b 


14.7 


9.6 


5.1 


4.3 


0.17 


Freshman English I 


0.0 


0.5 


-0.5 


0.5 


-0.06 


Freshman English II 


0.0 


0.0 


0.0 


NA 


NA 


First postprogram semester 












Enrolled in one or more English courses 3 












Developmental English 


51.3 


38.8 


12.5 ** 


5.7 


0.26 


Freshman English I 


14.1 


9.2 


4.9 


3.8 


0.17 


Freshman English II 


1.0 


1.6 


-0.6 


1.2 


-0.05 


Passed by end of semester 












Both English skills assessment tests b 


28.9 


19.7 


9.2 * 


5.0 


0.23 


Freshman English I 


10.6 


9.1 


1.4 


3.2 


0.05 


Freshman English II 


1.0 


1.6 


-0.6 


1.2 


-0.05 


Second postprogram semester 












Enrolled in one or more English courses 3 












Developmental English 


24.3 


28.3 


-4.0 


4.7 


-0.09 


Freshman English I 


14.7 


9.6 


5.1 


3.8 


0.17 


Freshman English II 


8.1 


9.1 


-1.0 


2.9 


-0.03 


Passed by end of semester 












Both English skills assessment tests b 


36.1 


27.6 


8.5 


5.3 


0.19 


Freshman English I 


22.3 


17.1 


5.2 


4.2 


0.14 


Freshman English II 


8.5 


9.7 


-1.1 


3.2 


-0.04 


Sample size (total = 385) 


197 


188 








Failed onlv one English test at baseline 












Program semester 












Enrolled in one or more English courses 3 












Developmental English 


88.6 


74.4 


14 2 *** 


3.4 


0.32 


Freshman English I 


8.9 


3.7 


5.2 ** 


2.2 


0.28 


Freshman English II 


0.0 


0.0 


0.0 


NA 


NA 



(continued) 
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Table 4.6 (continued) 





Program 


Control 




Standard 


Effect 


Outcome (%) 


Group 


Group 


Difference 


Error 


Size 


Passed by end of semester 












Both English skills assessment tests 


42.7 


32.5 


10.2 ** 


4.7 


0.22 


Freshman English I 


8.9 


3.1 


5.8 *** 


2.1 


0.33 


Freshman English II 


0.0 


0.0 


0.0 


NA 


NA 


First postprogram semester 












Enrolled in one or more English courses' 1 












Developmental English 


25.1 


29.9 


-4.9 


3.7 


-0.11 


Freshman English I 


30.1 


27.6 


2.4 


3.9 


0.05 


Freshman English II 


10.3 


5.9 


4.4 * 


2.4 


0.19 


Passed by end of semester 












Both English skills assessment tests b 


56.3 


44.5 


1 1 g *** 


4.5 


0.24 


Freshman English I 


31.4 


25.2 


6.3 


4.0 


0.14 


Freshman English II 


8.6 


5.1 


3.5 * 


2.0 


0.16 


Second postprogram semester 












Enrolled in one or more English courses 3 












Developmental English 


10.9 


17.5 


-6.6 ** 


2.9 


-0.17 


Freshman English I 


14.7 


14.0 


0.8 


2.6 


0.02 


Freshman English II 


19.1 


16.7 


2.4 


3.3 


0.06 


Passed by end of semester 












Both English skills assessment tests b 


59.2 


52.3 


6.9 


4.5 


0.14 


Freshman English I 


42.2 


35.5 


6.7 * 


4.0 


0.14 


Freshman English II 


23.1 


20.1 


3.0 


3.6 


0.07 


Sample size (total = 704) 


347 


357 








Passed both English tests at baseline 












Program semester 












Enrolled in one or more English courses 3 












Developmental English 


0.0 


0.0 


0.0 


NA 


NA 


Freshman English I 


90.8 


88.5 


2.4 


3.9 


0.07 


Freshman English II 


11.1 


12.3 


-1.2 


3.1 


-0.04 


Passed by end of semester 












Both English skills assessment tests 


100.0 


100.0 


0.0 


NA 


NA 


Freshman English I 


73.0 


68.9 


4.1 


4.5 


0.09 


Freshman English II 


8.9 


10.0 


-1.1 


2.7 


-0.04 


First postprogram semester 












Enrolled in one or more English courses 3 












Developmental English 


0.0 


0.0 


0.0 


NA 


NA 


Freshman English I 


12.4 


11.0 


1.4 


3.0 


0.04 


Freshman English II 


47.1 


43.1 


4.0 


4.1 


0.08 
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Table 4.6 (continued) 



Outcome (%) 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Passed by end of semester 












Both English skills assessment tests b 


100.0 


100.0 


0.0 


NA 


NA 


Freshman English I 


80.5 


74.5 


6.1 


3.8 


0.14 


Freshman English II 


46.0 


43.9 


2.1 


4.3 


0.04 


Second postprogram semester 












Enrolled in one or more English courses 3 












Developmental English 


0.0 


0.0 


0.0 


NA 


NA 


Freshman English I 


3.5 


3.2 


0.3 


1.6 


0.02 


Freshman English II 


14.2 


11.3 


2.9 


3.3 


0.09 


Passed by end of semester 












Both English skills assessment tests b 


100.0 


100.0 


0.0 


NA 


NA 


Freshman English I 


81.4 


75.8 


5.6 


4.0 


0.13 


Freshman English II 


56.2 


50.7 


5.5 


3.7 


0.11 


Sample size (total = 445) 


225 


220 









SOURCES: MDRC calculations from City University of New York skills assessment test data and 
Kingsborough Community College transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

‘'Kingsborough Community College's semesters comprise two sessions. Fall semesters include fall and 
winter sessions, and spring semesters include spring and summer sessions. In some cases, students took an 
English course in each session, resulting in two English courses for one semester. 

The skills assessment tests are City University of New York (CUNY)-wide and include test-taking 
attempts at any CUNY campus. 



on enrollment and passage of the regular English classes generally persist, although the im- 
pacts are only marginally statistically significant for English II in the first postprogram se- 
mester and for English I in the second postprogram semester. Importantly, however, the 
impacts are of nearly the same magnitude as the impacts observed after the program semes- 
ter. Given that these are cumulative measures, the fact that they do not increase suggests 
that most of the gains were obtained during the program semester. Notably, however, stu- 
dents in the program group were less likely to have enrolled in Developmental English in 
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the first and second postprogram semesters than one would expect, given the positive im- 
pact on passing the assessment tests. 

Finally, among those students who had passed both English tests at baseline, there 
are no observed gains in terms of English course progression for students in the program 
group. 



Overall, the results suggest that learning communities significantly improve a stu- 
dent’ s likelihood of passing the English assessment tests during the program semester. And, 
there is some evidence that learning communities improve a student's success in passing 
regular English courses as well. These impacts largely remain constant after the first post- 
program semester, suggesting that one-semester learning communities provide a one-time 
boost in educational attainment that then plateaus. 

General Educational Outcomes 

Given that nearly 10 percent of the study students enrolled in another institution 
(while not enrolled at Kingsborough) in at least one of the first four semesters after random 
assignment, data from institutions beyond Kingsborough are needed to fully understand the 
impact of learning communities on students’ educational outcomes. 

Table 4.7 presents results for a broader measure of educational attainment: persis- 
tence in college as measured by enrollment at Kingsborough and other postsecondary insti- 
tutions, using data from the National Student Clearinghouse. As discussed in Chapter 2, 
while the Clearinghouse data offer broad coverage of postsecondary institutions, their 
measures of a student’s educational attainment are quite limited. Specifically, these data 
indicate only enrollment (by type of institution) and degree attainment. 

As was evident from the transcript data from Kingsborough, the likelihood that a 
student — in either the program group or the control group — was enrolled in college de- 
clined with time. Thus, while over 85 percent of students were enrolled at any college dur- 
ing the learning communities semester, this proportion had fallen to just over 50 percent by 



71 




The Opening Doors Demonstration 
Table 4.7 

Enrollment Outcomes 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Program semester 












Enrolled in any college (%) 


87.3 


84.7 


2.6 


1.9 


0.07 


Enrolled in any 2-year college (%) 


87.1 


84.7 


2.4 


1.9 


0.07 


Enrolled in any 4-year college (%) 


0.4 


0.0 


0.4 * 


0.2 


N/A 


First nostnrogram semester 












Enrolled in any college (%) 


74.4 


71.2 


3.2 


2.2 


0.07 


Enrolled in any 2-year college (%) 


72.3 


70.3 


2.0 


2.2 


0.04 


Enrolled in any 4-year college (%) 


2.1 


1.0 


1.0 * 


0.6 


0.10 


Second nostnrogram semester 












Enrolled in any college (%) 


62.5 


61.1 


1.5 


2.4 


0.03 


Enrolled in any 2-year college (%) 


58.1 


57.0 


1.1 


2.5 


0.02 


Enrolled in any 4-year college (%) 


4.4 


4.1 


0.4 


1.1 


0.02 


Third nostnrogram semester 












Enrolled in any college (%) 


58.7 


53.0 


5.6 ** 


2.5 


0.11 


Enrolled in any 2-year college (%) 


52.0 


47.2 


4.9 * 


2.7 


0.10 


Enrolled in any 4-year college (%) 


7.2 


6.1 


1.0 


1.3 


0.04 


Any semester 1 












Enrolled in any college (%) 


91.2 


89.4 


1.8 


1.6 


0.06 


Enrolled in any 2-year college (%) 


90.7 


88.6 


2.1 


1.6 


0.07 


Enrolled in any 4-year college (%) 


8.7 


6.7 


2.1 


1.4 


0.08 


Number of semesters enrolled in any college 


2.8 


2.7 


0.1 * 


0.1 


0.09 


Sample size (total = 1,534) 


769 


765 









(continued) 
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Table 4.7 (continued) 



SOURCE: MDRC calculations from the National Student Clearinghouse data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

‘'Outcomes include data from the program semester through the third postprogram semester. 



the third semester after random assignment (or 1 .5 years after the Opening Doors Learning 
Communities semester). 13 

In each of the first three semesters after random assignment, students in the program 
group were only slightly more likely than those in the control group to enroll in any college 
or any two-year college. There is a suggestion that a higher proportion of them were more 
likely to enroll in a four-year college in the first semester after random assignment (the pro- 
gram semester) and in the second semester after random assignment, but the percentage of 
program group students who enrolled in a four-year college is so small that these differ- 
ences are not educationally important; the difference also disappears in the third semester 
after random assignment. 

Note that, in the third postprogram semester, students in the program group were 

5.6 percentage points more likely to be enrolled at any college and were 4.9 percentage 
points more likely to be enrolled at a two-year college — impacts that were also observed in 
the Kingsborough transcript data and that are statistically significant. In terms of degree 
attainment, there was generally no statistically detectable educational advantage of having 
been assigned to the program group (not shown in Table 4.7). That said, one may not have 
expected much of an impact on degree attainment only 1.5 years after random assignment. 

In general, using broader measures of student achievement in college, there is some 
evidence that learning communities may improve student persistence in college, as meas- 
ured by enrollment, over time. 

l3 The percentages of students registered at Kingsborough in each semester according to Table 4.3 do not ex- 
actly match the percentages of students enrolled at any college or even any two-year college according to Table 

4.7 due to the fact that some students in the study did not match to the Clearinghouse data, mostly because of non- 
existent or invalid Social Security numbers. 



73 




Impacts for Selected Subgroups 

Impacts on educational outcomes were examined for some different subgroups of 
students, defined using characteristics measured at baseline. The analyses found no mean- 
ingful differences in impacts according to the students’ immigrant status, their race or eth- 
nicity, whether they were the first in their family to attend college, the number of English 
assessment tests that they had passed at baseline, or whether they had a high school diploma 
or a General Educational Development (GED) certificate. The analysis for subgroups de- 
fined using gender shows that the magnitude of several of the impacts is larger for men than 
for women, but most of the differences between genders are not statistically significant. 
Appendix D presents the results for the gender-defined subgroups. 
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Chapter 5 

Social, Psychological, and Health Outcomes 



Community college reforms like the Opening Doors Learning Communities pro- 
gram that was implemented at Kingsborough Community College may come to affect the 
well-being of participating students over time. “Well-being” is broadly defined to include a 
range of social, psychological, and health outcomes. This chapter examines whether these 
learning communities had any effects on well-being among the Kingsborough sample at the 
12-month follow-up point. Following a baseline description of well-being for this sample, 
findings from analyses conducted to identify 12-month program impacts on these types of 
outcomes are presented. The chapter concludes with a brief discussion of the implications 
of these findings for future studies of community college reforms and student well-being. 

In sum, the key findings from these analyses are: 

• Students in the Kingsborough research sample were — in addition to being 
young — generally resilient and healthy at the start of the study, which de- 
creases the odds that the learning communities will have significant impacts 
on their social, psychological, and health outcomes over time. 

• Analyses of data from the Opening Doors 12-Month Survey produced little 
evidence to suggest that the learning communities had an impact on social, 
psychological, or health outcomes at the 12-month follow-up point. 



Opening Doors and Well-Being 

The fundamental goal of Opening Doors is to help community college students 
achieve better educational outcomes, like persistence in college, better grades, and degree 
completion. Some of these effects will be achieved in the near term (that is, during the first, 
or “program,” semester of Opening Doors Learning Communities), and others will become 
visible only with the passage of time (in postprogram semesters and after graduation). 

Early program impacts on educational outcomes — especially if they are initially 
strong or enduring — have the potential to influence not only educational attainment but 
also labor market outcomes (like employment, better jobs, and higher income) and indica- 
tors of well-being in the longer term. In sum, Opening Doors can act as a lever that helps 
students capitalize on the benefits of education and enjoy greater socioeconomic status 
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(SES) 1 and subsequently an enhanced sense of well-being. 2 The indicators of well-being 
that are examined in this chapter include positive social, psychological, and health out- 
comes, such as having a strong sense of self, better interpersonal relationships and greater 
social supports, more civic engagement and prosocial behavior, greater capacity to manage 
stress, healthier attitudes and behaviors, and — ultimately — better physical and mental 
health. 



Although these outcomes are of great interest, it is sensible to have low expectations 
about the degree to which Opening Doors Learning Communities might affect them during 
a 12-month period. As shown in the conceptual framework that guides the Opening Doors 
demonstration (see Chapter 1, Figure 1.2), the learning communities are theorized to have 
the greatest potential for influencing well-being through indirect effects, which emanate 
from a direct impact of the intervention on educational outcomes, which in turn can lead to 
greater SES. As discussed below, SES has consistently been shown to be a powerful deter- 
minant of well-being. This hypothesized causal chain of effects cannot fully evolve in a 12- 
month time frame, although the earliest signs of it might. Moreover, even direct effects of 
the learning community program on well-being — for example, positive changes in stu- 
dents’ self-esteem or peer networks stemming directly from their experience of the inter- 
vention — will also require considerable time before leading to observable changes in 
health behaviors. Likewise, changes in health behaviors typically do not lead to immediate 
effects on health. 

Therefore, expectations of finding impacts on well-being should not be high. How- 
ever, some outcomes may be more likely to change within the 12-month follow-up period 
of the survey. Therefore, three general kinds of well-being are distinguished and examined: 
(1) social and psychological well-being, (2) health behaviors, and (3) mental and physical 
health. As discussed above, outcomes in the first two categories are more susceptible to 
change than those in the third category, particularly within a 12-month time frame. 



Relevant Research 

The most compelling evidence to support the theorized relationships discussed 
above comes from the long-standing research literature that documents a strong positive 
association between education and health. Individuals with higher levels of education have 



SES is a multidimensional construct reflecting a person’s position in the class structure of a society. SES is 
typically assessed with measures of educational attainment, income, and occupational status or prestige, all of 
which are important to studies of health (Cockerham, 2007, page 63). 

2 lt is also possible that some community college reforms may have direct and independent effects on longer- 
term outcomes such as these, and these relationships are also examined. 
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lower mortality and morbidity rates from many types of disease 3 and display better health 
habits. For example, they demonstrate lower levels of smoking and binge drinking, and they 
also have a lower prevalence of obesity . 4 

However, it is important to emphasize that although there is clear evidence that 
higher levels of education are associated with enhanced well-being , 5 it is not clear whether 
this association represents a causal effect of education on these outcomes. Instead, it could 
be that factors that lead to greater success in school and higher educational attainment — 
such as having a wealthier or more supportive family or having had better health in child- 
hood — account for the observed relationship. The Opening Doors study is especially im- 
portant since, because of its randomized design, it can provide information on whether im- 
provements in educational opportunities can indeed improve social, psychological, and 
health outcomes. In short, to the extent to which the Opening Doors Learning Communities 
have early impacts on educational outcomes, the program has the potential to empirically 
demonstrate the subsequent effects of education on well-being one year after random as- 
signment. 

This focal relationship between education and health must also be understood as 
part of a more fundamental relationship between SES and health , 6 which is especially rele- 
vant to any discussion of the well-being of low-income populations. Research has consis- 
tently identified a “SES health gradient ,” 7 wherein improvements in components of SES — 
such as social class, education, income, and occupational prestige — are associated with 
better health. Therefore, although education — among indicators of SES — may be the 
most significant determinant of well-being, it should be viewed as part of this larger con- 
struct of SES . 8 This is important to consider, given the Opening Doors focus on low-income 
students, who — according to all measures of SES — have considerable ground to gain. 

In addition, education is believed to foster a greater sense of self and connectivity to 
others . 9 For example, greater educational attainment can strengthen and expand the social 
networks from which one can draw guidance, help, and support. It can also bring about 
greater awareness of social issues and civic engagement. Colleges and universities provide 
students with access to social support through relationships with fellow students, faculty, 

3 Lleras-Muney (2005). 

4 Christenson and Johnson ( 1 995); Deaton and Paxson (2001); Elo and Preston ( 1 996). 

5 Kolata (2007). 

6 Link and Phelan (1995). 

7 Adler and Newman (2002); Marmot and Wilkinson (1999). 

s Recent research suggests that education is more predictive of the onset of health problems, while income is 
more strongly associated with the progression of such problems (Herd, Goesling, and House, 2007). 

9 For research and commentary on the deleterious effects of social isolation on health, see Berkman and Glass 
(2000) and House (2001). 
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and staff, 10 some of whom are likely to become role models and mentors, and studies show 
that students who have mentors may have a greater sense of self-worth, be better able to 
weather personal crises, become more aware of educational and career opportunities, and 
ultimately set higher goals for themselves. 11 Moreover, to the extent that institutions suc- 
cessfully create a supportive and “bonding” environment, students — younger students es- 
pecially — may be less likely to engage in behaviors that compromise their own and their 
community’s well-being. 12 Learning communities are especially focused on building this 
type of connectivity. Moreover, studies illustrate that educational attainment is positively 
associated with broader social involvements, as indicated by effects on voting, volunteer- 
ism, newspaper readership, civic knowledge, and involvement in social groups and clubs. 13 
Thus, to the degree to which the Opening Doors Learning Communities program influences 
early educational outcomes, it may also affect some social and psychological indicators of 
well-being at the 12-month follow-up. 



Review of Relevant Findings on Educational Outcomes 

This chapter focuses on social, psychological, and health outcomes among partici- 
pants in Opening Doors Learning Communities. However, lessons from earlier analyses 
help to frame those outcomes presented here. 

As reported in Chapter 4, the program significantly influenced the courses that 
Kingsborough students took, as well as some educational outcomes, like passing classes 
during the program semester. However, those early effects on educational outcomes faded 
in subsequent semesters. Although the preceding analyses hint at the possibility of an en- 
during program impact on persistence in college, they do not offer strong evidence to sug- 
gest some of the indirect, longer-term effects of Opening Doors Learning Communities on 
well-being that are hypothesized above. 

More promising to the chapter at hand are the direct and positive effects of the 
Opening Doors Learning Communities program on the students’ overall college experience 
— in particular, on their enhanced sense of integration and belonging at Kingsborough. The 
program appears to have succeeded in creating a more supportive and bonding environment 



10 A large body of research suggests that people who enjoy strong and supportive relationships with others are 
better able to handle stressful life events and circumstances and, consequently, to preserve their emotional and 
physical well-being (Thoits, 1995; Turner and Turner, 1999). 

1 'Rhodes (2002). 

12 Eccles and Gootman (2001); Neumark-Sztainer, Stoiy, French, and Resnick ( 1997); Resnick et al. (1997). 
13 Dee (2004); Delli Carpini and Keeter (1996); Sullivan and Transue (1999); Uslaner, 2002. 
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for participating students, and these initial good feelings may be precursors to an enriched 
sense of social and psychological well-being that transcends the school context. 



Measures of Well-Being 

The measures of well-being that are examined in this chapter can be grouped into 
three general categories: (1) social and psychological outcomes; (2) health behaviors; and 
(3) mental and physical health, as illustrated below. 

Social and Psychological Outcomes 

• Outlook and identity: optimism, goal orientation, life engagement, and self- 
esteem and sense of self 

• Social support and civic engagement: general social support, having 
friends who value education, volunteerism, and civic engagement 

• Antisocial behavior: having spent time in reform school or prison (either the 
respondent or close friends) 

Health Behaviors 

• Smoking, binge drinking, illegal drug use, and having a relatively high num- 
ber of sexual partners 

Mental and Physical Health 

• Perceived stress, psychological distress, 14 self-rated health, and body mass 
index (BMI), which is based on a calculation using one’s height and weight 

Data for all these outcomes were collected via the Opening Doors 12-Month Sur- 
vey, and data for a subset of Kingsborough study participants were collected at baseline us- 
ing a briefer instrument. (Chapter 2 describes these data sources.) Some of these variables 
are represented with scale measures, which are summarized in Box 5.1 and detailed in Ap- 
pendix E. All other variable measures are described in the notes of the tables that follow. 



14 Scale measures of “nonspecific psychological distress” are designed to assess or “screen for” a range of 
broadly defined mental disorders — such as mood or anxiety disorders — rather than any one disorder in particu- 
lar (for example, major depression) (Kessler et al., 2002; Fmukawa, Kessler, Slade, and Andrews, 2003). 
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Box 5.1 



Behind the Scales Measuring Well-Being 

Outlook and Identity 

Optimism. To assess respondents’ level of optimism, they were asked the degree to which 
they agreed (on a scale of 1 to 4) with the following types of statements: In uncertain times, I 
usually expect the best; and Overall, I expect more good things to happen to me than bad. 

Goal orientation. To gauge respondents’ level of orientation in working toward their goals, 
they were asked the degree to which they agreed (on a scale of 1 to 4) with the following 
types of statements: I don’t think much about my long-term goals; and It is important for me 
to take time to plan out where I’m going in life. 

Life engagement. To gauge respondents’ level of engagement in life in general, they were 
asked the degree to which they agreed (on a scale of 1 to 4) with the following types of state- 
ments: I have lots of reasons for living; and I value my activities a lot. 

Self-esteem. To measure respondents’ level of self-esteem, they were asked the degree to 
which they agreed (on a scale of 1 to 4) with the following types of statements: I am able to 
do things as well as most other people; and I feel that I have a number of good qualities. 

Sense of self. To measure the degree to which respondents have a well-developed sense of 
themselves as individuals and in relation to significant others, they were asked the degree to 
which they agreed (on a scale of 1 to 4) with the following types of statements: There is at 
least one person who knows “the real you ”; You have a pretty good sense of the path you 
want to take in life and the steps to take to get there; You feel your life is filled with meaning, 
a sense of purpose; and People often seek your advice and support. 

Social Support and Civic Engagement 

General social support. To assess respondents’ perceived level of social support, they were 
asked the degree to which they agreed (on a scale of 1 to 4) with the following types of state- 
ments: There are people I know will help me if I need it; and I have a trustworthy person to 
turn to if I have problems. 

Friends value education. To gauge the degree to which respondents have friends who value edu- 
cational pursuits, they were asked the extent to which they felt that their friends supported their ef- 
forts to achieve educational goals as well as had educational goals of their own (on a scale of 1 to 

4) . This was done with the following types of statements: How supportive are your friends of you 
attending college? and Among your friends, how important is it to get good grades? 

Civic engagement. To measure respondents’ level of civic engagement, they were asked whether 
they had done things like registered to vote and donated time or money to a political campaign. 

Mental Health Outcomes 

Stress. To measure respondents’ level of stress, they were asked how often (on a scale of 1 to 

5) they felt, for example, unable to control the important things in life and that difficulties 
were piling up so high [that you] could not overcome them. 



Psychological distress. To measure respondents’ level of psychological distress, they were asked 
how often (on a scale of 0 to 4) they felt, for example, hopeless; restless or fidgety; and worthless. 
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Baseline Indicators of Social and Psychological Well-Being and 
Health 

The Opening Doors Learning Communities program was designed for first-time, in- 
coming freshmen who were attending school full time. As a consequence, this sample is 
young. About 95 percent were age 25 or younger at baseline. As shown in Table 5.1, they 
reported high levels of social support and low levels of stress and distress at random as- 
signment. For example, the cut point for meeting criteria for “serious psychological dis- 
tress” in the past 30 days according to Kessler’s K6 Screening Scale measure is a score of 
13 (the range is 0 to 24), and the average score in this sample is 4.60. Only 3.6 percent 
scored 13 or higher. 15 Similarly, only 3.4 percent of this sample reported that their overall 
health was “fair” or “poor” at baseline (as compared with “good,” “very good,” or “excel- 
lent”). Just over a third (33.9 percent) were either overweight or obese, according to the 
standard weight-status categories associated with the body mass index (BMI) measure. 
While this may seem high at first glance, it is considerably lower than the national average 
for U.S. adults (65 percent). 16 Thus, these baseline indicators collectively portray that the 
students in the Kingsborough research sample are — in addition to being young — gener- 
ally resilient and healthy, which decreases the odds that the Opening Doors Learning Com- 
munities program will have significant impacts on social, psychological, and health out- 
comes over time, simply because large improvements in health are not possible. 

Table 5.1 also demonstrates that there were few differences between the program 
and control groups in terms of students’ well-being at baseline. An important exception is 
that the program group had fewer current smokers than the control group at the start of the 
study (13.3 percent versus 19.1 percent). In addition, the program group was slightly less 
distressed according to their K6 scores. Consequently, both baseline smoking status and 
psychological distress are controlled in the 12-month impact models discussed next. In ad- 
dition, to be consistent with the methods used in Chapter 4, the analysis also controls for the 
Kingsborough students’ baseline status on the City University of New York (CUNY) skills 
assessment tests in reading and writing, which students are required to take before enroll- 
ment in the CUNY system. (Chapter 1 provides more information about the skills assess- 
ment tests, and Table 2.2 in Chapter 2 shows sample members’ performance on the tests.) 



15 This is about the same percentage as is seen in the general population of adults (National Health Interview 
Survey, 2007). 

16 Based on results from the National Health and Nutrition Examination Survey (National Center for Health 
Statistics, 2007). 
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The Opening Doors Demonstration 
Table 5.1 



Social Psychological and Health Measures at Baseline 
Kingsborough Community College Report 



Outcome 


Sample 

Size 


Full 

Sample 


Program 

Group 


Control 

Group 


General social support 3 ( 1 -4) 


1,411 


3.18 


3.18 


3.19 


Perceived stress' 3 (1-5) 


1,467 


2.34 


2.34 


2.35 


K6 score for psychological 
distress 3 (0-24) 


1,419 


4.60 


4.41 


4.78 * 


Indicator of high 
psychological distress (%) 


1,419 


3.6 


3.4 


3.8 


Health status fair or poor (%) 


1,512 


3.4 


3.7 


3.2 


Body mass index 3 (kg/m 2 ) 


1,432 


24.20 


24.18 


24.22 


Overweight or obese (BMI > 25) f (%) 


1,432 


33.9 


33.1 


34.6 


Current smoker (%) 


1,470 


16.2 


13.3 


19 1 *** 



SOURCE: MDRC calculations from the Opening Doors Baseline Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort. 

a 8-item scale about the presence of social support; response categories range from 1 = "strongly 
disagree" to 4 = "strongly agree." Items are averaged. 

b 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very 
often." Items are averaged. 

c 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

d Indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 
e BMI = weight in kilograms divided by height in meters squared. 

f l = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with 
BMI ranges for adults: underweight = <18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and 
obese = 30 or greater. 
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Twelve-Month Impacts on Social, Psychological, and Health 
Outcomes 

These analyses produced little evidence to suggest that Opening Doors Learning 
Communities had an impact on social or psychological outcomes at the 12-month follow-up 
point. Results for these outcomes are shown in Table 5.2. For only one indicator — a meas- 
ure of how strongly one’s friends value education — is there a statistically significant dif- 
ference between the program and control groups. The finding that survey respondents in the 
program group have friends who more strongly value education may be consistent with re- 
sults shown in Chapter 4, where it is reported that students in the program group have a 
greater sense of integration and belonging at Kingsborough. However, there are no statisti- 
cally significant differences between the two groups for any of the other social and psycho- 
logical indicators, including measures of outlook and identity, social support and civic en- 
gagement, and antisocial behavior. 

Table 5.3 demonstrates there are no significant differences in any of the health be- 
haviors and only one significant difference in an indicator of health. These data suggest a 
statistically significant effect on the indicator of high psychological distress, as measured by 
a K6 score in excess of 12. However, given that there is no statistically significant differ- 
ence in mean K6 scores between the program and control groups — as well as the fact that 
the two groups do not differ in perceived stress (the other mental health outcome) — this 
result concerning high psychological distress may be the product of chance rather than a 
true program impact. 



Implications for Future Research 

Overall, these analyses of the Opening Doors 12-Month Survey data do not indicate 
that the Opening Doors Learning Communities program in Kingsborough has improved 
students’ social, psychological, and health outcomes. This is not surprising. These are 
longer-term outcomes that not only take time to develop but also are contingent on the 
strength of earlier program impacts, and, as shown in Chapter 4, the learning communities, 
for the most part, have not had enduring effects on educational outcomes to date. Moreover, 
the baseline data on well-being of the Kingsborough sample clearly portray it as a resilient 
and healthy group among whom large improvements in most of these variables over time 
were very unlikely; consequently, the odds of uncovering program effects in these areas 
were low. 



81 




The Opening Doors Demonstration 
Table 5.2 

Social Psychological Outcomes 
Kingsborough Community College Report 

Sample Program Control Standard Effect 

Outcome Size Group Group Difference Error Size 



Outlook and identity 

Optimism 3 (1-4) 1,054 

Goal orientation 13 (1-4) 1,067 

Life engagement 0 (1-4) 1,063 

Self esteem d (1-4) 1,064 

Sense of self 3 (1-4) 1,064 

Social support and civic engagement 

General social support (1-4) 1,065 

Friends value education 8 (1-4) 1,010 

Did unpaid volunteer or 

community work in the past year (%) 1,075 

Civic engagement 1 ' (0-1) 1,064 

Antisocial behavior 

Spent time in reform school 

or prison in past year (%) 1,075 

Close friend spent time in reform 

school or prison in past year (%) 1,075 



2.97 


3.00 


-0.03 


0.0 


-0.07 


3.52 


3.48 


0.04 


0.0 


0.08 


3.46 


3.46 


0.00 


0.0 


0.00 


3.46 


3.50 


-0.04 


0.0 


-0.08 


3.52 


3.51 


0.01 


0.0 


0.03 



3.25 


3.24 


0.01 


0.0 


0.03 


3.35 


3.27 


0.08 ** 


0.0 


0.12 


22.9 


21.9 


1.0 


2.5 


0.02 


0.26 


0.26 


0.00 


0.0 


0.02 



3.3 


3.6 


-0.4 


1.1 


-0.02 


21.5 


23.3 


-1.8 


2.5 


-0.04 



(continued) 

Despite the lack of significant impacts in the 12-month impact study of the Opening 
Doors Learning Communities program at Kingsborough Community College, these social, 
psychological, and health outcome data remain a rich resource. In addition, significant 
program effects on well-being may be discovered in other sites in the Opening Doors demon- 
stration, especially where the target population is somewhat older and demonstrates greater 
variability in social, psychological, and health outcomes. (Table 1.1 in Chapter 1 describes the 



82 







Table 5.2 (continued) 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 
levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort, baseline psychological distress, baseline smoking 
status, and the number of English tests passed at baseline. Standard errors are clustered by learning 
community links. 

a 6-item scale about feelings of optimism; response categories range from 1 = "strongly disagree" to 
4 = "strongly agree." The six items are averaged. 

b 3-item scale about feeling focused on one's goals; response categories range from 1 = "strongly 
disagree" to 4 = "strongly agree." The three items are averaged. 

c 6-item scale about feelings that life is purposeful and worthwhile; response categories range from 
1 = "strongly disagree" to 4 = "strongly agree." The six items are averaged. 

d 4-item scale about feelings of self-esteem; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." The four items are averaged. 

e 13-item scale about feeling a strong sense of who one is, who one wants to be, and connections to 
others; response categories range from 1 = "strongly disagree" to 4 = "strongly agree." The thirteen 
items are averaged. 

f 

8-item scale about the presence of social support; response categories range from 1 = "strongly 
disagree" to 4 = "strongly agree." The eight items are averaged. 

g 6-item scale about the importance of education to friends; response categories range from 1 = 

"not very" to 4 = "extremely." The six items are averaged. 

h 4-item scale of activities indicative of civic engagement (registered to vote; voted in presidential 
election; donated time or money to a political campaign; attended a political speech, rally, or march). 
Each item is coded as a 0 ("no") or 1 ("yes"), and the four items are averaged. 



other sites, their programs, and their target groups.) Finally, nonexperimental analysis aimed 
at better understanding the nature of the relationship between well-being and key educa- 
tional outcomes or trajectories — outside the program context — might also hold interest- 
ing lessons for researchers and policymakers. 
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The Opening Doors Demonstration 
Table 5.3 

Health and Health Behaviors 

Kingsborough Community College Report 

Sample Program Control Standard Effect 

Outcome Size Group Group Difference Error Size 

Mental and physical health 



Perceived stress 3 (1-5) 


1,070 


2.24 


2.18 


0.06 


0.0 


0.08 


K6 score for psychological 
distress 1 * (0-24) 


1,066 


5.56 


5.35 


0.21 


0.2 


0.05 


Indicator of high 
psychological distress 1 * (%) 


1,066 


9.0 


4.9 


4 i ** 


1.7 


0.16 


Health status fair or poor (%) 


1,071 


14.2 


11.5 


2.7 


1.9 


0.08 


Body mass index 11 (kg/m 2 ) 


1,002 


24.41 


24.33 


0.09 


0.3 


0.02 


Overweight or obese (BMI > 25) e (%) 


1,002 


35.7 


36.8 


-1.1 


2.7 


-0.02 


Health behaviors 














Current smoker (%) 


1,072 


20.4 


20.7 


-0.3 


1.7 


-0.01 


Binge drinking in past month (%) 


1,069 


17.2 


19.9 


-2.7 


2.5 


-0.07 


Illegal drug use in past month (%) 


1,064 


10.7 


12.8 


-2.1 


1.8 


-0.06 


3 or more sexual partners in past month (%) 


1,010 


14.0 


15.2 


-1.2 


2.0 


-0.03 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort, baseline psychological distress, baseline smoking status, and 
the number of English tests passed at baseline. Standard errors are clustered by learning community links. 

a 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very 
often." Items are averaged. 

b 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

“indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 
d BMI = weight in kilograms divided by height in meters squared. 

e l = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with 
BMI ranges for adults: underweight = <18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and 
obese = 30 or greater. 



84 







Appendix A 

Supplementary Table for Chapter 2 




The Opening Doors Demonstration 
Appendix Table A.l 

Characteristics of Sample Members at Baseline 



Kingsborough Community College Report 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Gender (%) 


Male 


45.4 


42.2 


48.7 ** 


Female 


54.6 


57.8 


51.3 ** 


Age (%) 


17-18 years old 


44.5 


44.9 


44.1 


19-20 years old 


34.2 


35.8 


32.6 


21-34 years old 


21.3 


19.3 


23.3 * 


Average age (years) 


19.7 


19.6 


19.8 


Marital status (%) 


Married 


3.9 


3.7 


4.2 


Unmarried 


96.1 


96.3 


95.8 


Race/ethnicity (%) 


Hispanic/Latino 


20.4 


21.2 


19.6 


Black, non-Hispanic 


37.7 


38.1 


37.2 


White, non-Hispanic 


26.9 


24.5 


29.3 ** 


Asian or Pacific Islander 


8.6 


9.4 


7.8 


Other 


6.4 


6.8 


6.0 


One child or more in household (%) 


8.7 


8.3 


9.1 


Among sample members with children: 


Average age of youngest child (years) 


3.0 


2.7 


3.3 


Average household size (excluding roommates or boarders) 


3.8 


3.8 


3.7 


Household receiving any of the following benefits'* (%): 


Unemployment/Dislocated Worker benefits 


3.9 


2.3 


5.6 *** 


Supplemental Security Income (SSI) or disability 


10.4 


10.8 


10.0 


Cash assistance or welfare (TANF) 


5.7 


6.5 


4.9 


Food stamps 


9.0 


9.9 


8.0 


None of the above 


78.1 


78.5 


77.7 


Household in public or Section 8 housing (%) 


15.4 


16.6 


14.4 


Household receiving any government benefits (%) 


28.4 


28.8 


28.1 



(continued) 
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Appendix Table A.l (continued) 



Full Program Control 

Characteristic Sample Group Group 



Financially dependent on parents (%) 


74.2 


76.0 


72.4 


Ever employed (%) 


78.2 


76.4 


79.9 * 


Among those ever employed: 

Number of months employed at least 
halftime in the past year (%) 


None 


24.0 


24.9 


23.2 


1-3 months 


25.3 


26.9 


23.8 


4-6 months 


21.6 


20.6 


22.5 


7-9 months 


8.9 


8.8 


9.0 


10-12 months 


20.2 


18.8 


21.5 


Number of hours worked per week at current or last job (%) 


1-10 hours 


13.1 


13.3 


13.0 


11-20 hours 


23.3 


24.1 


22.4 


21-30 hours 


28.7 


28.3 


29.1 


31-40 hours 


25.9 


26.0 


25.8 


More than 40 hours 


9.0 


8.3 


9.7 


Average hourly wage at current or last job ($) 


7.6 


7.6 


7.6 


Currently employed (%) 


35.5 


34.1 


36.9 


Among those currently employed: 

Number of hours worked per week in current job (%) 


1-10 hours 


9.6 


9.5 


9.8 


11-20 hours 


25.4 


25.1 


25.6 


21-30 hours 


31.5 


30.9 


32.0 


31-40 hours 


25.0 


25.9 


24.2 


More than 40 hours 


8.5 


8.5 


8.4 


Average hourly wage at current job {$) 


7.8 


7.9 


7.8 


Respondent or household member receiving (%): 


Unemployment/Dislocated Worker benefits 


3.1 


2.1 


4.0 


Supplemental Security Income (SSI) or disability 


7.3 


7.6 


7.0 


Cash assistance or welfare (TANF) 


2.7 


2.3 


3.0 


Food stamps 


7.5 


8.9 


6.3 


Flighest grade completed (%) 


8 th grade or lower 


1.5 


1.4 


1.5 


9th grade 


4.3 


3.7 


4.9 


10 th grade 


7.9 


6.4 


9 4 ** 


1 1 th grade 


11.0 


11.0 


11.0 


12 th grade 


75.3 


77.4 


73.2 * 


Diplomas/degrees earned 3 (%) 


High school diploma 


70.9 


72.9 


68.8 * 


General Educational Development (GED) certificate 


28.6 


25.7 


31.4 ** 


Occupational/technical certificate 


2.0 


2.1 


2.0 



(continued) 
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Appendix Table A.l (continued) 



Characteristic 



Full Program Control 

Sample Group Group 



Date of high school graduation/GED receipt (%) 








During the past year 


70.2 


72.3 


68.0 * 


Between 1 and 5 years ago 


22.8 


21.4 


24.3 


More than 5 years ago 


7.0 


6.3 


7.7 


Main reason for enrolling in college”* (%) 








To complete a certificate program 


2.8 


3.3 


2.2 


To obtain an associate's degree 


29.7 


29.1 


30.3 


To transfer to a 4-year college/university 


50.2 


50.1 


50.3 


To obtain/update job skills 


10.8 


9.7 


11.9 


Other 


8.4 


10.0 


6.7 ** 


Completed any college courses/credits (%) 


7.3 


6.9 


7.8 


Among those who completed any college courses/credits: 








Average number of courses completed 


2.3 


1.8 


2.8 


First person in family to attend college (%) 


33.4 


34.9 


31.9 


Working personal computer in home (%) 


79.7 


79.8 


79.6 


Owns or has access to a working car (%) 


25.6 


25.2 


26.1 


Language other than English spoken regularly in home (%) 


46.9 


48.6 


45.2 


U.S. citizen (%) 


72.6 


72.4 


72.7 


Respondent born outside U.S. b (%) 


39.9 


42.3 


37.6 * 


Respondent or respondent's parent(s) born outside U.S. b (%) 


74.4 


76.0 


72.9 


Location of respondent's birthplace (%) 








North America 


60.0 


57.8 


62.2 * 


Asia 


6.3 


7.6 


4 9 ** 


Commonwealth of Independent States 0 


9.5 


8.4 


10.6 


Latin America and the Caribbean 


18.7 


20.6 


16.9 * 


Other 1 


5.5 


5.6 


5.4 


Location of respondent's mother's birthplace 0 (%) 








North America 


28.2 


26.3 


30.0 


Asia 


9.8 


11.1 


8.6 


Commonwealth of Independent States 0 


11.0 


9.7 


12.2 


Latin America and the Caribbean 


41.5 


43.0 


39.9 


Other 1 


9.6 


9.9 


9.3 


Sample size 


1,534 


769 


765 


(continued) 
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Appendix Table A.l (continued) 

SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort. 

Italics indicate nonexperimental data. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

a Distributions may not add to 100 percent because categories are not mutually exclusive. 
b "U.S." includes Puerto Rico. 

c This commonwealth includes Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, 
Moldova, Russia, Tajikistan, Turkmenistan (until August 2005), Ukraine, and Uzbekistan. 

d Other regions include the Baltic states, eastern and western Europe, North Africa, Sub-Saharan Africa, 
the Middle East, and Oceania. Countries are grouped by region according to the U.S. Bureau of the Census, 
International Data Base. 

e The majority of respondents (over 80 percent) reported that both parents were born in the same region. 
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Appendix B 

Survey Response Analysis 




This appendix reviews the derivation of the final research sample for the Opening 
Doors 12-Month Survey at Kingsborough Community College, provides the survey re- 
sponse rate, and assesses the potential for selection bias in survey response. 



Final Survey Sample and Survey Response Rate 

There were 1,534 sample members in the Opening Doors study at Kingsborough. 
The Opening Doors 12-Month Survey, which asked sample members questions about a 
wide range of topics — including their educational experiences, social relationships and 
supports, future outlook and identity, and health — was fielded for 1,520 sample members. 
Fourteen sample members were not attempted to be surveyed because they were ineligible, 
incarcerated, or incapacitated. A sample member who lived 50 miles beyond where the field 
interviewers were located and who did not have a telephone was considered ineligible. A 
sample member who was in the military and deployed outside the United States, had moved 
outside the United States, or was seriously injured in an accident and unable to be inter- 
viewed during the interview period was classified as incapacitated. 

Of the 1,520 surveys fielded, 1,111 sample members (73.0 percent) responded to 
the survey. Although MDRC received survey data for 1,111 sample members, 36 sample 
members (3.2 percent of the respondents) were dropped from the research sample because 
their interviews were conducted past the interview cutoff date. Many of the questions asked 
specifically about the respondent’s life during the previous 12 months, and those respon- 
dents who were interviewed more than 1 8 months after random assignment referred to peri- 
ods that did not correspond to the time period of interest. This left a final research sample of 
1,075 sample members and a response rate of 70.7 percent for the 12-month survey at 
Kingsborough. 



Assessment of Selective Survey Response 

Background Characteristics 

Appendix Table B.l compares the baseline characteristics of the survey respondent 
sample with the characteristics of the nonrespondent sample. The table indicates that fe- 
males were more likely than males to respond to the survey, accounting for 57.4 percent of 
the respondents and 48.7 percent of the nonrespondents. This difference is statistically sig- 
nificant, meaning that the detected differences are real and not due to chance. Additionally, 
the table indicates that blacks, high school graduates, and sample members receiving gov- 
ernment benefits at the point of random assignment were also more likely to respond to the 
survey. A joint test of the characteristics presented in the table shows that several of them 
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The Opening Doors Demonstration 
Appendix Table B.l 

Characteristics of Twelve-Month Survey Respondents and Nonrespondents 



Kingsborough Community College Report 



Characteristic 


Full 

Sample 


Respondents 


Nonrespondents 


Gender (%) 


Female 


54.9 


57.4 


48 7 *** 


Age (%) 


17-18 years old 


44.6 


46.2 


40.6 * 


19-20 years old 


34.0 


32.8 


37.1 


21-34 years old 


21.4 


21.0 


22.3 


Average age (years) 


19.7 


19.6 


19.8 


Marital status (%) 


Married 


4.0 


4.2 


3.5 


Race/ethnicity (%) 


Hispanic/Latino 


20.2 


19.2 


22.8 


Black, non-Hispanic 


37.7 


40.8 


30.4 *** 


White, non-Hispanic 


27.0 


25.4 


30.8 ** 


Asian or Pacific Islander 


8.6 


8.4 


9.0 


Other 


6.4 


6.2 


6.9 


Average household size (excluding 


roommates or boarders) 


3.8 


3.9 


3.6 ** 


Household receiving any government benefits (%) 


28.5 


30.8 


22.8 *** 


Financially dependent on parents (%) 


74.3 


75.4 


71.7 


Ever employed (%) 


78.1 


77.5 


79.4 


Currently employed (%) 


35.4 


33.7 


39.6 ** 


Highest grade completed (%) 


8th grade or lower 


1.4 


1.1 


2.0 


9th grade 


4.3 


4.0 


4.9 


10th grade 


8.0 


7.4 


9.5 


11th grade 


11.0 


10.3 


12.6 


12 th grade 


75.3 


77.2 


71.0 ** 


Diplomas/degrees earned 1 * (%) 


High school diploma 


71.1 


72.7 


67.0 ** 


General Educational Development (GED) certificate 


28.4 


26.8 


32.3 ** 


Occupational/technical certificate 


2.0 


1.9 


2.4 



(continued) 



94 







Appendix Table B.l (continued) 



Characteristic 


Full 

Sample 


Respondents 


Nonrespondents 


Date of high school graduation/GED receipt (%) 


During the past year 


70.1 


71.0 


67.9 


Between 1 and 5 years ago 


22.9 


22.5 


23.7 


More than 5 years ago 


7.1 


6.5 


8.4 


Main reason for enrolling in college* 1 (%) 


To complete a certificate program 


2.8 


2.6 


3.2 


To obtain an associate's degree 


29.8 


29.3 


31.0 


To transfer to a 4-year college/university 


50.2 


51.0 


48.4 


To obtain/update job skills 


10.8 


10.7 


11.2 


Other 


8.2 


8.4 


7.8 


Completed any college courses/credits (%) 


7.4 


8.0 


5.9 


Among those who completed 
any college courses/credits: 


Average number of courses completed 


2.3 


2.1 


3.2 


Language other than English 


spoken regularly in home (%) 


46.8 


45.7 


49.5 


U.S. citizen (%) 


72.6 


73.7 


69.9 


Respondent born outside U.S. b (%) 


39.9 


38.8 


42.6 


Respondent or respondent's parent(s) 


born outside U.S. (%) 


74.4 


73.5 


76.7 


Sample size 


1,520 


1,075 


445 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and research group. 

Italics indicate nonexperimental data. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

distributions may not add to 100 percent because categories are not mutually exclusive. 
b "U.S." includes Puerto Rico. 
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seem associated with students’ survey participation. (However, the results presented in the 
report do not qualitatively change when controlling for these characteristics.) 

Appendix Table B.2 compares the baseline characteristics of survey respondents in 
the program group with the characteristics of respondents in the control group to show 
whether respondents who had certain baseline characteristics are concentrated more in one 
research group than the other. The table shows that there are no statistically significant dif- 
ferences between the research groups regarding gender or receipt of government benefits. 
Blacks and high school graduates were also no more likely to be assigned to one research 
group or the other. Although the differences in these characteristics are shown as being sta- 
tistically significant in Appendix Table B.l, it is unlikely that these differences affected any 
analyses comparing research groups. 

However, the asterisks in the rightmost column of Appendix Table B.2 indicate a 
few statistically significant differences between the two research groups’ baseline character- 
istics. This suggests that there is the potential for some selection bias in the analyses that 
compare survey responses by research group. It is important to keep these differences in 
mind when looking at the impact analyses utilizing survey data. 

Academic Records 

MDRC received transcript data from Kingsborough for all 1,534 sample members 
in the study. If a sample member did not enroll at the college during a particular semester, 
there would still be a record for that individual, but the data fields would be empty. Thus, 
academic records data provide consistent and reliable information on academic performance 
at Kingsborough. 

As indicated in Appendix Table B.3, transcript data show that sample members who en- 
rolled at Kingsborough were more likely than nonenrollees to respond to the 12-month survey. 
This is not surprising, given that the school required updated contact information from such stu- 
dents. The updated information allowed the survey firm to track and interview these sample 
members more easily than sample members who were not enrolled at the college. 
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The Opening Doors Demonstration 

Appendix Table B.2 

Characteristics of Twelve-Month Survey Respondents, by Research Group 



Kingsborough Community College Report 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Gender (%) 


Female 


57.2 


59.4 


54.9 


Age (%) 


17-18 years old 


46.6 


46.3 


47.0 


19-20 years old 


32.7 


35.0 


30.2 * 


21-34 years old 


20.7 


18.7 


22.7 


Average age (years) 


19.6 


19.5 


19.7 


Marital status (%) 


Married 


4.1 


4.2 


4.1 


Race/ethnicity (%) 


Hispanic/Latino 


19.0 


19.7 


18.2 


Black, non-Hispanic 


40.3 


40.8 


39.8 


White, non-Hispanic 


26.1 


23.1 


29.2 ** 


Asian or Pacific Islander 


8.4 


9.9 


6.7 * 


Other 


6.2 


6.5 


6.0 


Average household size (excluding 


roommates or boarders) 


3.9 


4.0 


37 ** 


Household receiving any government benefits (%) 


30.3 


29.1 


31.5 


Financially dependent on parents (%) 


75.4 


78.2 


72.5 ** 


Ever employed (%) 


77.3 


75.5 


79.1 


Currently employed (%) 


33.7 


33.4 


34.1 


Highest grade completed (%) 


8 th grade or lower 


1.2 


1.3 


1.1 


9th grade 


4.2 


3.9 


4.4 


10 th grade 


7.3 


6.4 


8.2 


11 th grade 


10.1 


9.2 


11.0 


12 th grade 


77.3 


79.2 


75.3 


Diplomas/degrees earned 11 (%) 


High school diploma 


72.9 


75.0 


70.6 


General Educational Development (GED) certificate 


26.6 


23.9 


29.4 ** 


Occupational/technical certificate 


1.8 


1.8 


1.7 



(continued) 
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Appendix Table B.2 (continued) 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Date of high school graduation/GED receipt (%) 


During the past year 


71.4 


74.6 


68.1 ** 


Between 1 and 5 years ago 


22.0 


19.5 


24.5 * 


More than 5 years ago 


6.6 


5.9 


7.4 


Main reason for enrolling in college" 1 (%) 


To complete a certificate program 


2.6 


3.6 


1.7 * 


To obtain an associate's degree 


29.3 


27.7 


30.9 


To transfer to a 4-year college/university 


50.8 


50.3 


51.3 


To obtain/update job skills 


10.7 


10.5 


10.9 


Other 


8.7 


10.4 


6.9 ** 


Completed any college courses/credits (%) 


8.2 


7.3 


9.1 


Among those who completed 
any college courses/credits: 


Average number of courses completed 


2.1 


1.8 


2.4 


Language other than English 


spoken regularly in home (%) 


45.9 


48.3 


43.3 


U.S. citizen (%) 


73.8 


71.9 


75.7 


Respondent born outside U.S. b (%) 


38.9 


42.8 


35.0 *** 


Respondent or respondent's parent(s) 


born outside U.S. b (%) 


73.3 


75.1 


71.4 


Sample size 


1,075 


547 


528 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: A two-tailed t-test was applied to differences between the groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

d Distributions may not add to 100 percent because categories are not mutually exclusive. 
b "U.S." includes Puerto Rico. 
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The Opening Doors Demonstration 
Appendix Table B.3 
Transcript Outcomes: 

Twelve-Month Survey Respondents and Nonrespondents 
Kingsborough Community College Report 



Outcome (%) 


Full 

Sample 


Respondents 


Nonrespondents 


Registered for any courses 
during program semester 


92.3 


93.3 


89.8 ** 


Registered for any courses 
during first postprogram semester 


76.7 


80.1 


68.5 *** 


Registered for any courses 
during second postprogram semester 


60.8 


66.3 


47 4 *** 


Registered for any courses 
during third postprogram semester 


50.7 


56.2 


37.5 *** 


Sample size 


1,520 


1,075 


445 



SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical 
significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by number of English tests passed at baseline, research cohort, and 
research group. Standard errors are clustered by learning community links. 



Conclusion 

Survey responses were collected from close to 7 1 percent of the full research sam- 
ple at Kingsborough — a response rate that is suitable for a student population that tradi- 
tionally is mobile and difficult to track. Additionally, while analyses indicate some sources 
of potential bias, the number of sources is small and did not warrant any control in the out- 
come analyses. (Additional analyses that controlled for some of these characteristics did not 
qualitatively differ from those presented in this report.) Thus, while one should keep these 
caveats in mind while looking at the analyses, the Opening Doors 12-Month Survey should 
be considered a reliable measure of educational, social, and health outcomes for the Open- 
ing Doors study at Kingsborough. 
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Appendix C 

Description of Scales Presented in Chapter 4 




The following multi-item scale measures are presented in Chapter 4 and were cre- 
ated using data from the Opening Doors 12-Month Survey. Multi-item scales are useful for 
measuring complex constructs, such as those outlined below, because such constructs can- 
not be assessed easily with a single-item measure. All except one of these scale measures 
were created using questions that are included in the 2004 version of the Community Col- 
lege Survey of Student Engagement (CCSSE), a widely used assessment of student en- 
gagement. 1 The one exception is the “integration and sense of belonging at school” scale, 
which was created from survey questions developed for the Opening Doors demonstration. 
For each of these scales, a summary scale score is calculated and then divided by the num- 
ber of items that make up the scale, to create an average scale score. Finally, Cronbach’s 
alpha — an indicator of how well the items included in the scale measure a common under- 
lying construct 2 — is presented for each scale. 

Classroom and College Experiences 

Integration and Sense of Belonging at School (10-item scale; Cronbach’s alpha = .77) 

1. This is an unfriendly place, [responses were reversed in order to calculate the 
scale score] 

2. I do not feel that I fit in or belong at this campus, [responses were reversed] 

3. The instructors and staff understand who I am, where I am coming from. 

4. It is difficult to make good friends with other students, [responses were re- 
versed] 

5. The other students do not understand who I am, where I am coming from. 

6. This campus has the feeling of a community, where many people share the same 
goals and interests. 

7. Many people on this campus know me by name. 

8. I do not feel I am a part of campus life, [responses were reversed] 

9. I know my way around this place. 

10. I am proud to be a student here. 

Response categories: Strongly disagree (1) 

Disagree (2) 

Agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



'Community College Survey of Student Engagement (2007). 
2 Cronbach (1951). 
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Participation and Engagement (18-item scale; Cronbach’s alpha = .82) 

1 . Asked questions in class or contributed to class discussions. 

2. Made a class presentation. 

3. Prepared two or more drafts of a paper or assignment before turning it in. 

4. Worked on a paper or project that required integrating ideas or information from 
different classes. 

5. Came to class without completing readings or assignments, [responses were re- 
versed when calculating the scale score] 

6. Worked with other students on a project during class. 

7. Worked with classmates outside of class to prepare class assignments. 

8. Participated in a community -based project as part of a regular course. 

9. Used a listserve, chat group, Internet, etc. to discuss or complete an assignment. 

10. Used e-mail to communicate with an instructor. 

1 1 . Discussed grades or assignments with an instructor. 

12. Talked about career plans with an instructor or adviser. 

13. Discussed ideas from your readings or classes with instructors outside of class. 

14. Worked harder than you thought you could to meet an instructor’s standards or 
expectations. 

15. Worked with instructors on activities other than coursework. 

16. Discussed ideas from your readings or classes with others outside of class (stu- 
dents, family members, co-workers, etc.). 

17. Had serious conversations with students of a different race/ethnicity than your 
own. 

18. Had serious conversations with students who differ from you in terms of their 
religious beliefs, political opinions, or personal values. 

Response categories: Very often (1) 

Often (2) 

Sometimes (3) 

Never (4) 

Responses were summed and averaged. Scores range from 1 to 4. 
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Using Knowledge (Critical Thinking Curriculum) (6-item scale; Cronbach’s alpha = .78) 

1 . Analyzing the basic elements of an idea, experience, or theory. 

2. Synthesizing and organizing ideas, information, or experiences in new ways. 

3. Making judgments about the value or soundness of information, arguments, or 
methods. 

4. Applying theories or concepts to practical problems or in new situations. 

5. Using information you have read or heard to perform a new skill. 

6. Integrating ideas, information, or skills from different classes. 

Response categories: Very much (1) 

Quite a bit (2) 

Some (3) 

Very little (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Acquired Academic and Work Skills (16-item scale; Cronbach’s alpha = .92) 

1 . Acquiring a broad general education. 

2. Acquiring job or work-related knowledge and skills. 

3. Writing clearly and effectively. 

4. Speaking clearly and effectively. 

5. Thinking critically and analytically. 

6. Solving numerical problems. 

7. Using computing and information technology. 

8. Working effectively with others. 

9. Learning effectively on your own. 

10. Understanding yourself. 

11. Understanding people of other racial and ethnic backgrounds. 

12. Developing a personal code of values and ethics. 

13. Contributing to the welfare of your community. 

14. Developing clearer career goals. 

15. Gaining information about career opportunities. 

16. Developing a sense of confidence in your academic abilities. 

Response categories: Very much (1) 

Quite a bit (2) 

Some (3) 

Very little (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



105 




Appendix D 

Supplementary Tables for Chapter 4 




The Opening Doors Demonstration 
Appendix Table D.l 

Classroom and College Experiences of Sample Members, by Gender 



Kingsborough Community College Report 



Outcome 


Sample 

Size 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Male subs; roup 

Integration and sense of 
belonging at school a (%) 
Low 


413 


12.0 


14.2 


-2.2 


3.5 


-0.06 


High 


413 


16.2 


16.3 


-0.1 


3.5 


0.00 


Participation and engagement 11 (%) 
Low 


426 


20.3 


25.1 


-4.8 


4.0 


-0.11 


High 


426 


14.6 


11.3 


3.4 


3.6 


0.10 


Using knowledge (critical 
thinking curriculum) c (%) 
Low 


427 


11.9 


18.0 


-6.0 * 


3.4 


-0.16 


High 


427 


23.3 


19.4 


3.8 


3.8 


0.10 


Acquired academic and work skills'* (%) 
Low 


423 


15.2 


21.1 


-5.9 


3.6 


-0.15 


High 


423 


19.1 


16.4 


2.7 


3.6 


0.07 


Rated college experience 
good or excellent (%) 


428 


79.0 


76.6 


2.4 


4.0 


0.06 


Hours per week spent on 
campus in first semester 


460 


10.8 


10.9 


-0.1 


0.4 


-0.02 


Spent 12 or fewer hours per week 
on campus in first semester (%) 


460 


40.4 


36.3 


4.1 


4.6 


0.09 


Hours per week studying 
in first semester 


459 


7.5 


7.3 


0.1 


0.5 


0.02 


Studied 19 or more hours per week 
in first semester (%) 


459 


7.9 


8.6 


-0.7 


2.5 


-0.02 



(continued) 
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Appendix Table D.l (continued) 



Outcome 


Sample 

Size 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Female subgroup 

Integration and sense of 
belonging at school" (%) 
Low 


571 


10.7 


20.0 


_9 3 *** 


2.9 


-0.23 


High 


571 


15.6 


9.2 


6.4 ** 


2.7 


0.22 


Participation and engagement 11 (%) 
Low 


582 


11.2 


19.5 


-8 3 *** 


3.1 


-0.21 


High 


582 


20.4 


13.1 


7 3** 


3.2 


0.22 


Using knowledge (critical 
thinking curriculum) 11 (%) 
Low 


585 


12.3 


18.5 


-6.2 ** 


3.0 


-0.16 


High 


585 


25.4 


24.5 


0.8 


3.4 


0.02 


Acquired academic and work skills' 1 (%) 
Low 


582 


10.5 


16.0 


-5.5 * 


3.1 


-0.15 


High 


582 


22.2 


16.7 


5.5 * 


3.3 


0.15 


Rated college experience 
good or excellent (%) 


587 


86.1 


75.8 


10.3 *** 


3.4 


0.24 


Hours per week spent on 
campus in first semester 


614 


10.7 


10.9 


-0.2 


0.4 


-0.03 


Spent 1 2 or fewer hours per week 
on campus in first semester (%) 


614 


43.5 


39.7 


3.8 


3.8 


0.08 


Hours per week studying 
in first semester 


609 


7.6 


7.8 


-0.2 


0.5 


-0.03 


Studied 19 or more hours per week 
in first semester (%) 


609 


9.2 


9.5 


-0.3 


2.3 


-0.01 



(continued) 
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Appendix Table D.l (continued) 

SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels are 
indicated as: ttt = 1 percent; ft = 5 percent; f = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard errors 
are clustered by learning community links. 

a 10-item scale about sense of integration with and belonging to the school community; response categories 
range from 1 = "strongly disagree" to 4 = "strongly agree." "Low" is the percentage of sample members scoring 
one standard deviation below the mean; "high" is the percentage of sample members scoring one standard 
deviation above the mean. 

b 15-item scale about participation schoolwork, projects, and ideas; response categories range from 1 = "very 
often" to 4 = "never." "Low" is the percentage of sample members scoring one standard deviation below the mean; 
"high" is the percentage of sample members scoring one standard deviation above the mean. 

L 6-item scale about using aquired knowledge inside and outside the classroom; response categories range 
from 1 = "very much" to 4 = "very little." "Low" is the percentage of sample members scoring one standard 
deviation below the mean; "high" is the percentage of sample members scoring one standard deviation above the 
mean. 

d 16-item scale about aquiring academic and work skills, and a sense of self and community; response 
categories range from 1 = "very much" to 4 = "very little." "Low" is the percentage of sample members scoring 
one standard deviation below the mean; "high" is the percentage of sample members scoring one standard 
deviation above the mean. 
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The Opening Doors Demonstration 
Appendix Table D.2 

Transcript Outcomes, by Gender: Program Semester 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Male subsrouD 












Registered for any courses (%) 


91.9 


90.1 


1.8 


2.4 


0.06 


Enrolled in a learning community 1 * (%) 


83.7 


0.6 


83.1 *** 


2.8 


9.28 


Number of courses attempted 


4.7 


4.2 


0.5 *** 


0.2 


0.24 


Number of credits attempted 


15.3 


14.5 


0.8 


0.5 


0.13 


Regular credits 


9.6 


9.8 


-0.3 


0.4 


-0.05 


Equated credits 


5.7 


4.7 


1.0 ** 


0.4 


0.23 


Passed all courses (%) 


40.5 


27.3 


13 2 *** 


3.6 


0.30 


Number of courses passed 


3.6 


2.9 


q 7 *** 


0.1 


0.35 


Withdrew from any courses (%) 


27.7 


37.1 


_g 3 *** 


3.3 


-0.19 


Number of course withdrawals 


0.6 


0.7 


- 0.2 ** 


0.1 


-0.14 


Number of credits earned 


10.9 


9.3 


1 7 *** 


0.5 


0.23 


Regular credits 


7.6 


7.1 


0.5 


0.4 


0.08 


Equated credits 


3.3 


2.1 


1 2 *** 


0.4 


0.33 t 


Term GPA (%) 












No GPA b 


10.3 


12.5 


-2.3 


2.5 


-0.07 


3.0 to 4.0 or B/A 


30.3 


24.4 


5.9 


3.9 


0.14 


2.0 to 2.9 or C/B- 


32.2 


25.1 


7 0 ** 


3.4 


0.16 t 


1.0 to 1.9 orD/C- 


15.6 


20.6 


-5.0 * 


2.8 


- 0.12 


0 to 0.9 or F/D- 


11.8 


17.4 


-5.6 ** 


2.6 


-0.15 


Sample size (total = 697) 


325 


372 








Female subgroun 












Registered for any courses (%) 


94.1 


92.4 


1.7 


1.9 


0.06 


Enrolled in a learning community** (%) 


86.7 


0.5 


86.2 *** 


2.4 


12.10 


Number of courses attempted 


5.0 


4.6 


0.3 ** 


0.1 


0.19 


Number of credits attempted 


16.0 


15.8 


0.2 


0.4 


0.03 


Regular credits 


10.3 


10.5 


-0.3 


0.4 


-0.05 


Equated credits 


5.7 


5.3 


0.5 


0.3 


0.10 


Passed all courses (%) 


45.1 


38.3 


6.8 * 


3.5 


0.14 


Number of courses passed 


4.0 


3.4 


0.5 *** 


0.1 


0.25 



(continued) 
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Appendix Table D.2 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Withdrew from any courses (%) 


25.7 


32.0 


- 6.2 * 


3.2 


-0.13 


Number of course withdrawals 


0.5 


0.7 


- 0.1 


0.1 


- 0.11 


Number of credits earned 


12.0 


11.4 


0.6 


0.5 


0.09 


Regular credits 


8.4 


8.3 


0.1 


0.4 


0.02 


Equated credits 


3.6 


3.1 


0.5 


0.3 


0.12 


Term GPA (%) 












No GPA b 


7.5 


11.1 


-3.7 * 


2.1 


- 0.12 


3.0 to 4.0 or B/A 


37.7 


30.7 


7 0 ** 


3.4 


0.15 


2.0 to 2.9 or C/B- 


28.9 


30.7 


- 1.8 


3.0 


-0.04 


1.0 to 1 .9 or D/C- 


14.7 


14.5 


0.2 


2.3 


0.01 


0 to 0.9 or F/D- 


11.2 


13.0 


-1.7 


2.2 


-0.05 


Sample size (total = 837) 


444 


393 









SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance 
levels are indicated as: ttt = 1 percent; tt = 5 percent; f = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. 
Standard errors are clustered by learning community links. 

''While most students were enrolled in all three linked courses, a small number withdrew from part of 
the link. 

b The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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The Opening Doors Demonstration 

Appendix Table D.3 

Transcript Outcomes, by Gender: 

First, Second, and Third Postprogram Semesters 

Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 




Male subsroun 

First postprogram semester 


Registered for any courses (%) 


77.6 


71.8 


5.8 * 


3.1 


0.13 




Number of credits attempted 


12.4 


11.3 


1.1 * 


0.6 


0.13 




Regular credits 


10.0 


9.0 


1.0 ** 


0.5 


0.15 




Equated credits 


2.4 


2.3 


0.0 


0.4 


0.01 




Number of credits earned 


8.3 


7.4 


0.8 


0.6 


0.12 




Regular credits 


7.2 


6.3 


0.9 * 


0.5 


0.15 




Equated credits 


1.1 


1.1 


-0.1 


0.2 


-0.03 




Term GPA (%) 


No GPA a 


24.6 


29.3 


-4.6 


3.2 


-0.10 




3.0 to 4.0 or B/A 


18.9 


19.5 


-0.6 


2.8 


-0.01 




2.0 to 2.9 or C/B- 


26.9 


20.8 


6.1 * 


3.2 


0.15 




1.0 to 1.9 orD/C- 


13.3 


15.8 


-2.4 


2.7 


-0.07 




0 to 0.9 or F/D- 


16.2 


14.6 


1.6 


2.7 


0.05 




Second postprogram semester 


Registered for any courses (%) 


62.4 


54.4 


8.0 ** 


3.5 


0.16 


tt 


Number of credits attempted 


9.9 


8.8 


1.1 * 


0.6 


0.13 


t 


Regular credits 


8.6 


7.5 


1.2 ** 


0.6 


0.15 


t 


Equated credits 


1.3 


1.3 


0.0 


0.2 


-0.01 




Number of credits earned 


6.7 


6.1 


0.6 


0.6 


0.08 




Regular credits 


6.2 


5.6 


0.5 


0.5 


0.08 




Equated credits 


0.6 


0.5 


0.1 


0.1 


0.04 


t 


Term GPA (%) 


No GPA a 


38.9 


46.7 


_7 g ** 


3.6 


-0.16 


tt 


3.0 to 4.0 or B/A 


15.6 


15.4 


0.1 


2.9 


0.00 




2.0 to 2.9 or C/B- 


21.0 


16.6 


4.4 


3.2 


0.12 




1.0 to 1.9 orD/C- 


12.1 


13.9 


-1.7 


2.6 


-0.05 




0 to 0.9 or F/D- 


12.4 


7.4 


5.0 ** 


2.0 


0.19 





(continued) 
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Appendix Table D.3 (continued) 





Program 


Control 




Standard 


Effect 


Outcome 


Group 


Group 


Difference 


Error 


Size 


Third postprogram semester 












Registered for any courses (%) 


51.9 


42.8 


9.1 ** 


3.8 


0.18 


Number of credits attempted 


7.7 


6.5 


1.2 * 


0.6 


0.14 


Regular credits 


7.0 


5.8 


1.2 ** 


0.6 


0.17 


Equated credits 


0.6 


0.7 


-0.1 


0.2 


-0.03 


Number of credits earned 


5.6 


4.7 


0.9 * 


0.5 


0.14 


Regular credits 


5.4 


4.4 


0.9 * 


0.5 


0.15 


Equated credits 


0.3 


0.3 


0.0 


0.1 


-0.02 


Term GPA (%) 












No GPA a 


49.3 


58.0 


-8.6 ** 


3.7 


-0.17 


3.0 to 4.0 or B/A 


14.6 


15.0 


-0.4 


2.7 


-0.01 


2.0 to 2.9 or C/B- 


17.8 


12.7 


5.2 * 


2.7 


0.16 


1.0 to 1.9 orD/C- 


11.6 


8.5 


3.1 


2.2 


0.11 


0 to 0.9 or F/D- 


6.7 


5.9 


0.8 


1.8 


0.03 


Sample size (total = 697) 


325 


372 








Female subgroup 












First postprogram semester 












Registered for any courses (%) 


77.4 


77.9 


-0.4 


2.9 


-0.01 


Number of credits attempted 


12.6 


12.2 


0.3 


0.5 


0.04 


Regular credits 


10.3 


10.1 


0.2 


0.5 


0.03 


Equated credits 


2.3 


2.1 


0.1 


0.3 


0.04 


Number of credits earned 


8.9 


8.5 


0.4 


0.4 


0.06 


Regular credits 


7.7 


7.5 


0.2 


0.4 


0.03 


Equated credits 


1.2 


1.0 


0.2 


0.2 


0.08 


Term GPA (%) 












No GPA a 


24.8 


23.7 


1.1 


3.0 


0.03 


3.0 to 4.0 or B/A 


25.4 


26.2 


-0.8 


3.1 


-0.02 


2.0 to 2.9 or C/B- 


25.1 


23.3 


1.7 


2.9 


0.04 


1.0 to 1.9 orD/C- 


13.1 


15.0 


-2.0 


2.3 


-0.05 


0 to 0.9 or F/D- 


11.7 


11.8 


-0.1 


2.0 


0.00 


Second postprogram semester 












Registered for any courses (%) 


60.8 


63.4 


-2.6 


3.4 


-0.05 


Number of credits attempted 


9.9 


10.4 


-0.5 


0.6 


-0.06 


Regular credits 


oo 

O O 


9.1 


-0.2 


0.6 


-0.03 


Equated credits 


1.0 


1.4 


-0.3 * 


0.2 


-0.11 


Number of credits earned 


7.4 


7.7 


-0.3 


0.5 


-0.04 


Regular credits 


6.9 


6.9 


0.0 


0.5 


-0.01 


Equated credits 


0.5 


0.8 


-0.3 ** 


0.1 


-0.13 



(continued) 
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Appendix Table D.3 (continued) 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Term GPA (%) 












No GPA a 


40.6 


37.4 


3.2 


3.4 


0.07 


3.0 to 4.0 or B/A 


19.2 


24.9 


-5.6 ** 


2.9 


-0.13 


2.0 to 2.9 or C/B- 


23.2 


17.6 


5.6 ** 


2.6 


0.15 


1.0 to 1.9 orD/C- 


9.1 


12.1 


-2.9 


2.4 


-0.09 


0 to 0.9 or F/D- 


7.9 


8.1 


-0.2 


1.8 


-0.01 


Third postprogram semester 












Registered for any courses (%) 


53.8 


52.4 


1.4 


3.4 


0.03 


Number of credits attempted 


8.0 


7.8 


0.2 


0.6 


0.03 


Regular credits 


7.2 


7.1 


0.1 


0.5 


0.01 


Equated credits 


0.8 


0.7 


0.1 


0.1 


0.07 


Number of credits earned 


6.2 


6.1 


0.1 


0.5 


0.02 


Regular credits 


5.8 


5.8 


0.0 


0.5 


0.01 


Equated credits 


0.4 


0.3 


0.1 


0.1 


0.06 


Term GPA (%) 












No GPA a 


48.2 


49.4 


-1.2 


3.5 


-0.02 


3.0 to 4.0 or B/A 


18.2 


19.8 


-1.6 


2.7 


-0.04 


2.0 to 2.9 or C/B- 


19.7 


18.4 


1.3 


2.6 


0.03 


1.0 to 1.9 orD/C- 


8.2 


8.3 


-0.1 


1.9 


0.00 


0 to 0.9 or F/D- 


5.6 


4.1 


1.5 


1.3 


0.08 


Sample size (total = 837) 


444 


393 









SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance 
levels are indicated as: ttt = 1 percent; tf = 5 percent; t = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. 
Standard errors are clustered by learning community links. 

a The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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The Opening Doors Demonstration 

Appendix Table D.4 

Cumulative Outcomes, by Gender: 

Program Semseter Through Third Postprogram Semester 

Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Male subgroup 












Registered for any courses (%) 


94.3 


93.9 


0.4 


1.9 


0.02 


Number of semesters registered 


2.8 


2.6 


0.2 ** 


0.1 


0.19 t 


Number of credits earned 


31.6 


27.6 


4.0 ** 


1.7 


0.17 


Regular credits 


26.4 


23.5 


2.9 * 


1.6 


0.14 


Equated credits 


5.2 


4.1 


1.2 * 


0.6 


0.20 


Cumulative GPA a (%) 












No GPA b 


8.5 


9.5 


-1.1 


2.2 


-0.04 


3.0 to 4.0 or B/A 


21.4 


20.3 


1.0 


3.0 


0.03 


2.0 to 2.9 or C/B- 


33.4 


28.1 


5.4 


3.3 


0.12 


1.0 to 1.9 orD/C- 


20.4 


22.0 


-1.6 


2.9 


-0.04 


0 to 0.9 or F/D- 


16.4 


20.1 


-3.8 


3.0 


-0.09 


Sample size (total = 697) 


325 


372 








Female subgroup 












Registered for any courses (%) 


95.9 


94.4 


1.5 


1.6 


0.07 


Number of semesters registered 


2.9 


2.9 


0.0 


0.1 


0.00 


Number of credits earned 


34.5 


33.7 


0.8 


1.6 


0.04 


Regular credits 


28.9 


28.5 


0.3 


1.5 


0.02 


Equated credits 


5.6 


5.1 


0.5 


0.5 


0.07 


Cumulative GPA a (%) 












No GPA b 


6.4 


8.5 


-2.1 


1.9 


-0.08 


3.0 to 4.0 or B/A 


28.1 


28.0 


0.1 


3.1 


0.00 


2.0 to 2.9 or C/B- 


34.1 


30.9 


3.2 


3.0 


0.07 


1.0 to 1.9 orD/C- 


19.2 


19.0 


0.3 


2.6 


0.01 


0 to 0.9 or F/D- 


12.1 


13.5 


-1.4 


2.3 


-0.04 


Sample size (total = 837) 


444 


393 
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Appendix Table D.4 (continued) 



SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels 
are indicated as: ttt = 1 percent; tt = 5 percent; f = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard 
errors are clustered by learning community links. 

a Cumulative GPA is based on credit-bearing courses taken from random assignment through the end of the 
third postprogram semester. Courses in which students did not receive a passing grade and subsequently 
repeated are not included in the cumulative GPA, as per Kingsborough Community College policy. 

b The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 
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The Opening Doors Demonstration 
Appendix Table D.5 

English Skills Assessment Test Outcomes, by Gender 
Kingsborough Community College Report 





Program 


Control 




Standard 


Effect 


Outcome (%) 


Group 


Group 


Difference 


Error 


Size 


Male subgroup 












Program semester 












Attempted reading test 


11.7 


5.4 


6.3 ** 


2.7 


0.28 ft 


Passed reading test by end of semester 


85.0 


78.9 


6.1 ** 


2.4 


0.15 ff 


Attempted writing test 


36.4 


22.0 


24 4 *** 


4.5 


0.35 f 


Passed writing test by end of semester 


46.9 


38.0 


g 9 *** 


3.4 


0.18 


Attempted either English skills assessment test 


41.5 


25.8 


2g 7 *** 


4.7 


0.36 ff 


Passed both English skills 
assessment tests by end of semester 


46.3 


37.5 


8.8 ** 


3.4 


0.18 


Program and postprogram semesters 3 












Attempted reading test 


15.4 


9.4 


6.0 *** 


2.3 


0.21 


Passed reading test by end 
of second postprogram semester 


88.8 


83.5 


5.3 ** 


2.3 


0.14 f 


Attempted writing test 


47.1 


35.7 


224 *** 


3.8 


0.24 


Passed writing test by end 
of second postprogram semester 


60.4 


51.8 


8.6 ** 


3.9 


0.17 


Attempted either English skills assessment test 


50.7 


38.2 


22 5 *** 


3.5 


0.26 f 


Passed both English skills assessment tests 
by end of second postprogram semester 


60.1 


51.8 


8.3 ** 


3.9 


0.17 


Sample size (total = 697) 


325 


372 








Female subgroup 












Program semester 












Attempted reading test 


14.0 


13.2 


0.7 


2.7 


0.02 


Passed reading test by end of semester 


82.9 


83.4 


-0.4 


2.0 


- 0.01 


Attempted writing test 


36.0 


29.4 


6.6 * 


3.8 


0.14 


Passed writing test by end of semester 


58.4 


54.8 


3.6 


3.1 


0.07 


Attempted either English skills assessment test 


41.4 


35.9 


5.5 


4.1 


0.11 



(continued) 
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Appendix Table D.5 (continued) 



Outcome (%) 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Passed both English skills 
assessment tests by end of semester 


57.1 


54.0 


3.1 


3.0 


0.06 


Program and postprogram semesters 3 












Attempted reading test 


19.4 


17.0 


2.4 


1.9 


0.07 


Passed reading test by end 
of second postprogram semester 


88.4 


88.4 


-0.1 


1.8 


0.00 


Attempted writing test 


46.5 


42.4 


4.1 


2.9 


0.08 


Passed writing test by end 
of second postprogram semester 


70.8 


68.3 


2.5 


3.1 


0.05 


Attempted either English skills assessment test 


50.2 


45.8 


4.4 


2.9 


0.09 


Passed both English skills assessment tests 
by end of second postprogram semester 


69.3 


67.3 


2.0 


3.1 


0.04 


Sample size (total = 837) 


444 


393 









SOURCE: MDRC calculations from City University of New York skills assessment test data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels are 
indicated as: ftt = 1 percent; ft = 5 percent; f = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard errors 
are clustered by learning community links. 

a Outcomes include data from the program semester through the second postprogram semester. 
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The Opening Doors Demonstration 
Appendix Table D.6 
Enrollment Outcomes, by Gender 
Kingsborough Community College Report 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Male suberoun 












Program semester 












Enrolled in any college (%) 


87.5 


85.1 


2.3 


2.7 


0.07 


Enrolled in any 2-year college (%) 


87.5 


85.1 


2.3 


2.7 


0.07 


Enrolled in any 4-year college (%) 


0.3 


0.0 


0.3 


0.3 


N/A 


First postprogram semester 












Enrolled in any college (%) 


75.6 


69.2 


6.5 ** 


3.2 


0.14 


Enrolled in any 2-year college (%) 


73.8 


68.3 


5.5 * 


3.2 


0.12 


Enrolled in any 4-year college (%) 


1.8 


1.1 


0.7 


0.9 


0.07 


Second postprogram semester 












Enrolled in any college (%) 


62.8 


58.5 


4.3 


3.5 


0.09 


Enrolled in any 2-year college (%) 


59.2 


53.9 


5.3 


3.6 


0.11 


Enrolled in any 4-year college (%) 


3.6 


4.6 


- 1.0 


1.6 


-0.05 


Third postprogram semester 












Enrolled in any college (%) 


56.8 


49.0 


77 ** 


3.7 


0.15 


Enrolled in any 2-year college (%) 


51.2 


43.7 


7.5 * 


3.9 


0.15 


Enrolled in any 4-year college (%) 


6.8 


5.4 


1.4 


1.8 


0.06 


Any postprogram semester 3 












Enrolled in any college (%) 


91.4 


91.1 


0.4 


2.2 


0.01 


Enrolled in any 2-year college (%) 


91.4 


90.5 


0.9 


2.3 


0.03 


Enrolled in any 4-year college (%) 


8.0 


6.5 


1.5 


2.0 


0.06 


Number of semesters enrolled in any college 


2.8 


2.6 


0.2 ** 


0.1 


0.15 


Sample size (total = 697) 


325 


372 
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Appendix Table D.6 (continued) 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


Effect 

Size 


Female subgroup 












Program semester 












Enrolled in any college (%) 


87.2 


84.2 


3.0 


2.6 


0.08 


Enrolled in any 2-year college (%) 


87.0 


84.2 


2.8 


2.6 


0.08 


Enrolled in any 4-year college (%) 


0.4 


0.0 


0.4 


0.3 


N/A 


First postprogram semester 












Enrolled in any college (%) 


73.6 


73.0 


0.6 


3.0 


0.01 


Enrolled in any 2-year college (%) 


71.4 


72.0 


-0.6 


3.1 


-0.01 


Enrolled in any 4-year college (%) 


2.2 


1.1 


1.2 


0.8 


0.12 


Second postprogram semester 












Enrolled in any college (%) 


62.6 


63.1 


-0.5 


3.2 


-0.01 


Enrolled in any 2-year college (%) 


57.7 


59.5 


-1.9 


3.4 


-0.04 


Enrolled in any 4-year college (%) 


5.0 


3.6 


1.4 


1.5 


0.07 


Third postprogram semester 












Enrolled in any college (%) 


60.2 


56.7 


3.4 


3.2 


0.07 


Enrolled in any 2-year college (%) 


52.7 


50.3 


2.4 


3.4 


0.05 


Enrolled in any 4-year college (%) 


7.4 


6.9 


0.5 


1.8 


0.02 


Any postprogram semester 3 












Enrolled in any college (%) 


91.0 


87.8 


3.2 


2.2 


0.10 


Enrolled in any 2-year college (%) 


90.1 


86.7 


3.4 


2.3 


0.10 


Enrolled in any 4-year college (%) 


9.2 


6.9 


2.3 


1.9 


0.09 


Number of semesters enrolled in any college 


2.8 


2.8 


0.1 


0.1 


0.05 


Sample size (total = 837) 


444 


393 
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Appendix Table D.6 (continued) 

SOURCE: MDRC calculations from the National Student Clearinghouse data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels are 
indicated as: f ft = 1 percent; tt - 5 percent; t = 10 percent. 

Estimates are adjusted by research cohort and the number of English tests passed at baseline. Standard errors 
are clustered by learning community links. 

a Outcomes include data from the program semester through the second postprogram semester. 
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Appendix E 

Description of Scales Presented in Chapter 5 




The following multi-item scale measures are presented in Chapter 5 and were cre- 
ated using data from the Opening Doors 12-Month Survey. Multi-item scales are useful for 
measuring complex constructs, such as those outlined below, because such constructs can- 
not be assessed easily with a single-item measure. Most of these scale measures have been 
widely used in related literature, and footnotes are added to reference the original source 
from which scales were drawn or adapted. Three measures — “sense of self,” “friends value 
education,” and “civic engagement” — were developed for the Opening Doors demonstra- 
tion. 1 The measures of “civic engagement” and “psychological distress” are coded as sum- 
mative scales, which means that the values assigned to each response are added together to 
create a summary scale score. For the remaining measures, a summary scale score is calcu- 
lated and then divided by the number of items that make up the scale, to create an average 
scale score. Finally, Cronbach’s alpha — an indicator of how well the items included in the 
scale measure a common underlying construct 2 — is presented for each scale. 



Social and Psychological Outcomes 

Outlook and Identity 

Optimism 3 (6-item scale; Cronbach’s alpha = .60) 4 

1. In uncertain times, I usually expect the best, [responses were reversed in or- 
der to calculate the scale score] 

2. If something can go wrong for me, it will. 

3. I am always optimistic about my future. 

4. I hardly ever expect things to go my way. [responses were reversed] 

5. I rarely count on good things happening to me. [responses were reversed] 

6. Overall, I expect more good things to happen to me than bad. 

Response categories: Strongly disagree ( 1 ) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



'Questions included in these new measures are similar to those used elsewhere. 

2 Cronbach (1951). 

3 Scheier, Carver, and Bridges (1994). 

4 Data on a subset of measures are available at both baseline and the 12-month follow-up. All Cron- 
bach’s alphas shown above were calculated using 12-month follow-up data. 
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Goal Orientation 5 (3-item scale; Cronbach’s alpha = .59) 

1 . I don’t think much about my long-term goals. 

2. I have many long-term goals that I will work to achieve. 

3. It is important for me to take time to plan out where I’m going in life. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Life Engagement 6 (6-item scale, Cronbach’s alpha = .76) 

1 . There is not enough purpose in my life, [responses were reversed in order to 
calculate the scale score] 

2. I don’t care very much about the things I do. [responses were reversed] 

3. To me, the things I do are all worthwhile. 

4. I have lots of reasons for living. 

5. Most of what I do seems trivial and unimportant to me. [responses were re- 
versed] 

6. I value my activities a lot. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



5 For a discussion of the measures of “reactive responding,” see Psychosocial Working Group of the 
MacArthur Research Network on Socioeconomic Status and Health (2007). See also Taylor and Seeman 
(1999). 

6 Scheier et al. (2006). 
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Self-Esteem 7 (4-item scale; Cronbach’s alpha = .62) 

1 . I am able to do things as well as most other people. 

2. I feel that I’m a person of worth, or at least on an equal basis with others. 

3. I feel that I have a number of good qualities. 

4. I take a positive attitude toward myself. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Sense of Self (13-item scale; Cronbach’s alpha = .85) 

1 . Your goals in life are becoming clearer. 

2. People know they can count on you to “be there” for them. 

3. You have a clear sense of your beliefs and values. 

4. There is at least one person who knows “the real you.” 

5. You have a good deal of freedom to explore things in life that interest you. 

6. You feel respected by others as an adult. 

7. There is at least one person with whom you can talk about anything. 

8. You feel that you are important, that you “matter,” to other people. 

9. You have a pretty good sense of the path you want to take in life and the 
steps to take to get there. 

10. You can envision the kind of person you'd like to become. 

11. You feel your life is filled with meaning, a sense of purpose. 

12. It is easy for you to make close friends. 

13. People often seek your advice and support. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



7 Adapted from Rosenberg (1965). 



129 




Social Support and Civic Engagement 

General Social Support 8 (8-item scale; Cronbach’s alpha = .74) 

1 . There are people I know will help me if I need it. 

2. There is no one I feel comfortable talking about problems with, [responses 
were reversed in order to calculate the scale score] 

3. I am with a group of people who think the same way I do about things. 

4. If something went wrong, no one would help me. [responses were reversed] 

5. I have a trustworthy person to turn to if I have problems. 

6. I do not think that other people respect what I do. [responses were reversed] 

7. There is no one who likes to do the things I do. [responses were reversed] 

8. There are people who value my skills and abilities. 

Response categories: Strongly disagree (1) 

Disagree (2) 

Agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Friends Value Education (6-item scale; Cronbach’s alpha = .87) 

1 . How supportive are your friends of you attending college? 

2. Among your friends, how important is it to. . .Go to college? 

3 . ... Get good grades ? 

4. . . .Complete a college degree or training program? 

5. . . .Use a college degree or program certificate to get a better job? 

6. . . .Pursue advanced study after college? 

Response categories: Not very (1) 

Somewhat (2) 

Quite a bit (3) 

Extremely (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



8 Adapted from Cutrona and Russell (1987). 
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Civic Engagement (4-item summative scale; Cronbach’s alpha = .54) 

1 . Are you registered to vote? 

2. Did/do you plan to vote in the 2004 presidential election? 

3. Since [date of random assignment], have you donated your time or money 
to a political campaign? 

4. Since [date of random assignment], have you attended a political speech, 
rally, or march? 

Each item has two response categories (1 = Yes and 0 = No). The four items 
are added together and divided by 4. Scores range from 0 to 1 . 



Health Outcomes 

Mental Health 

Stress 9 (4-item scale; Cronbach’s alpha = .66) 

1. In the last 30 days.. .How often have you felt you were unable to control the 
important things in your life? 

2. ...How often have you felt confident about your ability to handle your per- 
sonal problems? [responses were reversed in order to calculate the scale 
score] 

3. ...How often have you felt that things were going your way? [responses 
were reversed] 

4. . . .How often have you felt difficulties were piling up so high that you could 
not overcome them? 

Response categories: None of the time (1) 

A little of the time (2) 

Some of the time (3) 

Most of the time (4) 

All of the time (5) 

Responses were summed and averaged. Scores range from 1 to 5. 



’Adapted from Cohen, Kamarck, and Mermelstein (1983); Cohen and Williamson (1988). 
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Psychological Distress 10 (6-item summative scale; Cronbach’s alpha = .77) 

1 . During the past 30 days. . .about how often did you feel nervous? 

2. . . .did you feel hopeless? 

3. . . .did you feel restless or fidgety? 

4. . . .did you feel so depressed that nothing could cheer you up? 

5. . . .did you feel that everything was an effort? 

6. . . .did you feel worthless? 

Response categories: None of the time (0) 

A little of the time (1) 

Some of the time (2) 

Most of the time (3) 

All of the time (4) 

Responses were summed and averaged. Scores range from 0 to 24, with a cut 
point of 13 to determine nonspecific psychological distress. 



10 Kessler et al. (2002). 
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Earlier MDRC Publications on Opening Doors 



Helping Low-Wage Workers Persist in Education Programs 

Lessons from Research on Welfare Training Programs and Two Promising Community College 
Strategies 

2008. Lashawn Richburg-Hayes 

Enhancing Student Sendees at Lorain County Community College 
Early Results from the Opening Doors Demonstration in Ohio 
2007. Susan Scrivener, Jenny Au 

Enhancing Student Sendees at Owens Community College 
Early Results from the Opening Doors Demonstration in Ohio 
2007. Susan Scrivener, Michael Pih 

A Whole ’Nother World 

Students Navigating Community College 

2006. Alissa Gardenhire-Crooks, Herbert Collado, Barbara Ray 

Paying for Persistence 

Early Results of a Louisiana Scholarship Program for Low-Income Parents Attending 
Community College 

2006. Thomas Brock, Lashawn Richburg-Hayes 
Building Learning Communities 

Early Results from the Opening Doors Demonstration at Kingsborough Community College 
2005. Dan Bloom, Colleen Sornmo 

Promoting Student Success in Community College and Beyond 
The Opening Doors Demonstration 

2005. Thomas Brock, Allen LeBlanc with Casey MacGregor 
Support Success 

Services That May Help Low-Income Students Succeed in Community College 
2004. Rogeair Purnell, Susan Blank with Susan Scrivener, Reishma Seupersad 

Changing Courses 

Instructional Innovations That Help Low-Income Students Succeed in Community College 
2003. Richard Kazis, Marty Liebowitz 

Money Matters 

How Financial Aid Affects Nontraditional Students in Community Colleges 
2003. Victoria Choitz, Rebecca Widorn 

Supporting CalWORKs Students at California Community Colleges 
An Exploratory Focus Group Study 
2003. Laura Nelson, Rogeair Purnell 



Opening Doors 

Students' Perspectives on Juggling Work, Family, and College 
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2002. Lisa Matus-Grossman, Susan Gooden with Melissa Wavelet. Melisa Diaz. Reishma 
Seupersad 

Welfare Reform and Community Colleges 
A Policy and Research Context 

2002. Thomas Brock. Lisa Matus-Grossman, Gayle Hamilton 
Opening Doors to Earning Credentials 

Impressions of Community College Access and Retention from Low-Wage Workers 
2001. Lisa Matus-Grossman, Susan Tinsley Gooden 

Opening Doors 

Expanding Educational Opportunities for Low-Income Workers 
Published with the National Governors Association Center for Best Practices 
2001. Susan Golonka, Lisa Matus-Grossman 



NOTE: All MDRC publications are available for free download at www.mdrc.org. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy ar- 
eas and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 



Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




